-- Logs begin at Fri 2022-08-05 04:23:22 UTC, end at Fri 2022-08-05 06:08:12 UTC. -- Aug 05 04:59:00.538112 dv-cinder-815582-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 04:59:00.548799 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 04:59:00 2022] *** Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: nodename: dv-cinder-815582-14-iscsi Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: machine: x86_64 Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: clock source: unix Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: pcre jit disabled Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: detected number of CPU cores: 4 Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: current working directory: / Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: detected binary path: /usr/bin/uwsgi-core Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: your processes number limit is 47680 Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: your memory page size is 4096 bytes Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: detected max file descriptor number: 2048 Aug 05 04:59:00.550815 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: lock engine: pthread robust mutexes Aug 05 04:59:00.551669 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: thunder lock: enabled Aug 05 04:59:00.551669 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 04:59:00.551669 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 04:59:00.589865 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: Python main interpreter initialized at 0x55ffc86019c0 Aug 05 04:59:00.589865 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: python threads support enabled Aug 05 04:59:00.589865 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: your server socket listen backlog is limited to 100 connections Aug 05 04:59:00.589865 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: your mercy for graceful operations on workers is 90 seconds Aug 05 04:59:00.590238 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 04:59:00.590398 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: *** Operational MODE: preforking *** Aug 05 04:59:00.590588 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: *** uWSGI is running in multiple interpreter mode *** Aug 05 04:59:00.590588 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: spawned uWSGI master process (pid: 80367) Aug 05 04:59:00.590783 dv-cinder-815582-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 04:59:00.591824 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: spawned uWSGI worker 1 (pid: 80378, cores: 1) Aug 05 04:59:00.591824 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: spawned uWSGI worker 2 (pid: 80379, cores: 1) Aug 05 04:59:00.591824 dv-cinder-815582-14-iscsi devstack@keystone.service[80367]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 04:59:05.936539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 04:59:05.936539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Aug 05 04:59:05.937415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 04:59:05.937415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: from cryptography.utils import int_from_bytes Aug 05 04:59:05.937534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 04:59:05.937534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: from cryptography.utils import int_from_bytes Aug 05 04:59:05.937534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 04:59:05.937534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: from cryptography.utils import int_from_bytes Aug 05 04:59:05.985515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 04:59:05.985704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 04:59:05.985843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] command line args: [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 04:59:05.985937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 04:59:05.986143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 04:59:05.986274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 04:59:05.986355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ================================================================================ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 04:59:05.986429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] command line args: [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 04:59:05.986558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.986641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 04:59:05.986754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ================================================================================ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 04:59:05.986835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_dir = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.986946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] admin_token = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] config_source = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config_dir = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] debug = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] config_source = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] debug = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.987948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_config_append = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.988970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_dir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_config_append = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_dir = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.989956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.990913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991218 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.991932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.992983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] publish_errors = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] publish_errors = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.993929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] standard_threads = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.994925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] standard_threads = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_journal = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_json = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.995904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_stderr = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] use_syslog = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_journal = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_json = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_stderr = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.996743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] use_syslog = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 04:59:05.996921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.997977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.998975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:05.999913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.000888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.001900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.002958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.003915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.004933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.005865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.006940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.007955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.008921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.009969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.010956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.011935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.012936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.013038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.013189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.013339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.013517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.013683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.013829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.014936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.015100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.015241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.015418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.015583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.015742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.015918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.016908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.017916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.018964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.019899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.020932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.external = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.021991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.password = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.external = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] auth.token = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.022985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.password = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] auth.token = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.023919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.024931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025850 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.025964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.026987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.027933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.028959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.029970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.030892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.031931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.032987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.033995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.034896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.035948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.036948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.037958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.038959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.039920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.040940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.041951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.042063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.042173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.042289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.042439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.042795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.042920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.043960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.044990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.045981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.046976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.047889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.048903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.049888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.050901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.051951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.052883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.053004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.053116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.053226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.053337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.053447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.053557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.054996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.055924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.056945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.057936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.058967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.059917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.060925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.061928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.062981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.063893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.064919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.065949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.066950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.067892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.068999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80378) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 04:59:06.069796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.069796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.070302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.070302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.070519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.070629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.070797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 04:59:06.070970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80379) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 04:59:06.099837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.099837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.099837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.100237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.100237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.100518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.106689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.109737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.136723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.136942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.142029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.142268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.188501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.188904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.237506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.237751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.238434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.238657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.300275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.300464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.300708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.300832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.300933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.301892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.324326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.324586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.324711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.324871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.324955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.325894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.336463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.336920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.349988 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80378) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.350811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80379) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 04:59:06.367201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.367414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 04:59:06.367512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.367732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 04:59:06.367883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.368039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 04:59:06.368122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.368186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.368352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 04:59:06.368477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.368627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.368711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.368876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.369001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.369140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.369348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.378471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.378714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.378999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.379121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.379256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.379362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.379490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.379806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.386087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.386674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.390776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.391057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.391335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.391458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.391588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.391681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.391909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.392934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.393059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.393200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.393295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.393419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.393497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.393754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.394017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.415937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.416777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.417398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.417674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.418267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.418591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.422295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.422569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.422839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.423181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.423459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.423734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.429739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.430011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.430680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.430978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.434091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.434378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.434652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.434942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.435066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.435224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.435427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.435560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.435695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.435793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.435946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.436091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.436237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.436510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.436778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.437054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.451723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.451955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.452907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.453184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.455809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.457124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.458931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.459228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.460298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.460592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.463563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.464960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.466753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.468307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.469889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.470180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.470475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.470766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.471038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.471311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.471451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.471739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.472046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.472333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.472614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.472890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.487093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.487375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.487636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.487930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.488772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.489052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.489322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.489595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.494914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.496506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.496720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.496720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.498092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.498376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.499576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.499877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.500163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.500630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.500906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.501216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.501511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.501803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.502086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.502365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.512502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.512748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.513050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.513325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.513598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.513871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.514150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.518161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.518576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.518985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.519393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.519818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.520242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.520678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.529713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.532671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.535552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.538120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.539694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.540001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.540281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.545634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.546639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.548257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.549509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.551144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.551417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.551698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.551991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.553322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.557006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.559056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.559465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.559910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.560363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.560646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.560917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.566908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.567661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80378) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.567957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.568233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.568500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.568622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.568759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.569016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.569283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.569569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.569950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.570218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.570494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.570760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.571005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.571258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80378) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.571415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.571707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.572015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.572290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.580336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.580636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.580915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.587757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80379) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 04:59:06.588061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.588346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.588618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.588885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.589156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.589431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.589703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.590095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.590438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.590697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.590963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.591240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.591500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80379) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 04:59:06.604186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ffc86019c0 pid: 80378 (default app) Aug 05 04:59:06.607553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad2eab-cb42-4a33-98d0-e5f063ebe572 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:06.607867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad2eab-cb42-4a33-98d0-e5f063ebe572 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:06.608130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad2eab-cb42-4a33-98d0-e5f063ebe572 None None] PATH_INFO: `/v3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:06.613052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Fri Aug 5 04:59:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:06.624822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ffc86019c0 pid: 80379 (default app) Aug 05 04:59:14.380207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:14.380761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:14.380761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:14.382008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 2/2] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:14.392079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:14.392816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:14.392932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:14.472769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: INFO dbcounter [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] Registered counter for database keystone Aug 05 04:59:14.494102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG oslo_db.sqlalchemy.engines [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80379) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 04:59:14.506076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writer thread running {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 04:59:14.689882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG passlib.handlers.bcrypt [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80379) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 04:59:14.690299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG passlib.handlers.bcrypt [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80379) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 04:59:14.750436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:14.753131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:14.902206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1/3] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 04:59:15.045111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:15.120050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: INFO dbcounter [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] Registered counter for database keystone Aug 05 04:59:15.144932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG oslo_db.sqlalchemy.engines [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80378) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 04:59:15.156905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writer thread running {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 04:59:15.278960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca516c6a-35bd-4df7-9dfe-801eac69ea3a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:15.284404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:15.285039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:15.285322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:15.285595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:15.294427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:15.345521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 04:59:15.345835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee18789-c81f-4cfd-bbe7-9ab9aafb022c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:15.356018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 3/4] 10.222.0.30 () {58 vars in 1324 bytes} [Fri Aug 5 04:59:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 04:59:15.363305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:15.386808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb11670-eb0e-4c81-b271-c965f108be2f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:15.393455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:15.394574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:15.394886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:15.395163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] PATH_INFO: `/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:15.397030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] RBAC: Authorizing `identity:get_project(project_id=4ea26c15401c49d28fd797d4ee85aa6c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:15.447279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 04:59:15.447630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66e566-943b-4c94-baf3-2154186bd946 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:15.452393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 2/5] 10.222.0.30 () {56 vars in 1207 bytes} [Fri Aug 5 04:59:15 2022] GET /identity/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:17.407936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d39bc-dafb-4fce-9e8d-4853389bb710 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:17.408505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d39bc-dafb-4fce-9e8d-4853389bb710 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:17.408505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d39bc-dafb-4fce-9e8d-4853389bb710 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:17.414972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 4/6] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:17.422111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:17.422555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:17.422645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:17.444781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:17.446289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:17.454886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 3/7] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 05 04:59:17.486757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d39bc-dafb-4fce-9e8d-4853389bb710 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:17.502118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d39bc-dafb-4fce-9e8d-4853389bb710 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:17.507475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6977d-88d3-48f6-9a5b-4512e49a8d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:17.508140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6977d-88d3-48f6-9a5b-4512e49a8d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:17.508420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6977d-88d3-48f6-9a5b-4512e49a8d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:17.508693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6977d-88d3-48f6-9a5b-4512e49a8d72 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:17.516679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6977d-88d3-48f6-9a5b-4512e49a8d72 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:17.518472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6977d-88d3-48f6-9a5b-4512e49a8d72 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:17.527312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 5/8] 10.222.0.30 () {58 vars in 1324 bytes} [Fri Aug 5 04:59:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 04:59:17.533238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:17.546405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9edee7-918c-4709-a151-c41448d0210e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:17.552201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401d4ac-2f43-4748-be3c-cfa57f865a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:17.552831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401d4ac-2f43-4748-be3c-cfa57f865a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:17.553114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401d4ac-2f43-4748-be3c-cfa57f865a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:17.553385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401d4ac-2f43-4748-be3c-cfa57f865a01 admin admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:17.555251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401d4ac-2f43-4748-be3c-cfa57f865a01 admin admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:17.557124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401d4ac-2f43-4748-be3c-cfa57f865a01 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:17.558656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 4/9] 10.222.0.30 () {56 vars in 1198 bytes} [Fri Aug 5 04:59:17 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:19.585354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda665-38f9-4c0c-be7b-9ea59c2836ca None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:19.585922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda665-38f9-4c0c-be7b-9ea59c2836ca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:19.585922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda665-38f9-4c0c-be7b-9ea59c2836ca None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:19.587248 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 6/10] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:19.592714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:19.593159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:19.593295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:19.609369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045457c-12c1-4384-a5d7-938113c0af62 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:19.609626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045457c-12c1-4384-a5d7-938113c0af62 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:19.609977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045457c-12c1-4384-a5d7-938113c0af62 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:19.611337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 7/11] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:19.614124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:19.615329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:19.616668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:19.616955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:19.617232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:19.661006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80378) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 04:59:19.661432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG passlib.handlers.bcrypt [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80378) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 04:59:19.698539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 5/12] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:19.706925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:19.708585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:19.721213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:19.745363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:19.807948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 8/13] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:19.823969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb3e97-d00e-4c2c-8892-6497e1f3b92e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:19.829339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:19.861785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:19.862367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:19.862645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:19.862952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:19.873868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:19.882742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:19.883653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:19.884783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 6/14] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:19.890150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:19.931515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d31df-c7a6-4cdb-bc95-015795fbf341 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:19.933837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:19.968036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:19.968926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:19.969231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:19.969498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:19.972870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:19.991750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e836-ccc6-4adf-8d52-d54389e533e9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:20.028845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:20.029458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:20.029754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:20.030030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:20.030766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:20.031067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.032510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.038426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.039974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.044103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 7/15] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:20.049724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:20.139965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 9/16] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:20.146027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:20.148994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:20.149105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0387e04-abc8-43ff-89ce-0592ad794eea None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:20.253814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43c62d-8fd1-4323-9e05-29e0b620964a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:20.292954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:20.293843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:20.294120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:20.294222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:20.294392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:20.294845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:20.295134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:20.295415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:20.296600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.298048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.299633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 8/17] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:20 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:20.305499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:20.313453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.314939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.320306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 04:59:20.333201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:20.334072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:20.334852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 04:59:20.334852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 04:59:20.334852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 04:59:20.334852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:20.334852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:20.340489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:20.396955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:20.405758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9019d-d16a-4b50-bbdb-15bc3b11f3d4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:20.439131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:20.439805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:20.440112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:20.440389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:20.441468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09daf2-192f-44b5-aa11-284f43504b1a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:20.441571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.443011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.444637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:20.450529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:20.474998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:20.475634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:20.475945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:20.476215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:20.476905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:20.477208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.478602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea804c87-0e99-4c00-a80a-03fdde82bb21 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.495520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 04:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:20.532297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:20.557889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9db77-1aee-49a9-bd4b-0a3178541d41 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:20.599423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:20.600124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:20.600372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:20.600619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:20.601436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.domain.id=default, user.id=c8b08b7805764c45aae169c453594d3d) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:20.601748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:20.603290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713140f-9aab-463b-8ff1-514c3e01ba05 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:20.611305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1322 bytes} [Fri Aug 5 04:59:20 2022] GET /identity/v3/role_assignments?user.id=c8b08b7805764c45aae169c453594d3d&scope.domain.id=default&role.id=47aca6aac2934a5cb816872abfc94994 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:22.821184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88957179-9b62-4248-a176-29ce8a5c2fea None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:22.821728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88957179-9b62-4248-a176-29ce8a5c2fea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:22.821728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88957179-9b62-4248-a176-29ce8a5c2fea None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:22.823498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:22.825097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23016d7-b64d-4829-90d9-4f6a5298c7c8 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:22.825473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23016d7-b64d-4829-90d9-4f6a5298c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:22.825667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23016d7-b64d-4829-90d9-4f6a5298c7c8 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:22.827029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 11/23] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:22.829348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:22.829859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:22.830319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:22.841116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:22.841680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:22.842138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:22.868832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:22.871188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:22.872539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:22.872892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:22.984703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 12/24] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:23.003472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 13/25] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:23.016212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.090405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.134200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:23.146477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:23.193614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bf60-3f6f-49c2-b3a9-93bc349f752c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:23.211930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c9aa5-882b-4633-bdea-ee735b67b35f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:23.227816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33757b-0966-44e4-b165-a0292e39e936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:23.228480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33757b-0966-44e4-b165-a0292e39e936 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.228757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33757b-0966-44e4-b165-a0292e39e936 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.229031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33757b-0966-44e4-b165-a0292e39e936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.239514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33757b-0966-44e4-b165-a0292e39e936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-fb33757b-0966-44e4-b165-a0292e39e936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:23.244703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:23.245536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:23.246755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 13/26] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:23.248021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbfd04-6c49-49f6-a2df-335fc942ea39 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.248305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbfd04-6c49-49f6-a2df-335fc942ea39 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.248580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbfd04-6c49-49f6-a2df-335fc942ea39 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.250098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 14/27] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:23.255718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbfd04-6c49-49f6-a2df-335fc942ea39 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.260447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:23.263203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.263490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.263803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.285944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:23.287453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b22c301c-e9a3-4d9a-a666-6c3976b3ab64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 04:59:23.315205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:23.316117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:23.316882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 04:59:23.316882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 04:59:23.316882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 04:59:23.316882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:23.328163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 14/28] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:23.329572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.329854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.330136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.363620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:23.364800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:23.394268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbfd04-6c49-49f6-a2df-335fc942ea39 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:23.431702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:23.432373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.432652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.432925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.433705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:23.434030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:23.435421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:23.436956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:23.446087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 15/29] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:23.446977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.549648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c033-a536-4940-8790-d59db7f11384 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:23.573339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 15/30] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:23.574007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.585080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:23.585721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.585999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.586283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.586973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:23.587276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:23.588694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0232ba9-595e-4967-9269-e2d0b6e543f0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:23.606319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 16/31] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 04:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:23.607676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810f11-9a47-4f90-a474-9eec007ae7d4 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.607983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810f11-9a47-4f90-a474-9eec007ae7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.608261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810f11-9a47-4f90-a474-9eec007ae7d4 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.609721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 17/32] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:23.611723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810f11-9a47-4f90-a474-9eec007ae7d4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.626943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:23.675150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:23.725561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810f11-9a47-4f90-a474-9eec007ae7d4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:23.761602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:23.762266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.762547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.762858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.763565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:23.765065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:23.818086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b05b07-7e88-4fe7-ab5a-33527f039de6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:23.854178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239f79c-f068-4f26-9071-354a9a190c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:23.854926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f79c-f068-4f26-9071-354a9a190c33 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.855123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f79c-f068-4f26-9071-354a9a190c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.855424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f79c-f068-4f26-9071-354a9a190c33 None admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.857016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239f79c-f068-4f26-9071-354a9a190c33 None admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:23.858484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239f79c-f068-4f26-9071-354a9a190c33 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:23.860197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 16/33] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:23 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:23.861308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:23.861593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:23.861871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:23.884561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:23.886771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:23.914383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 18/34] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 04:59:23 2022] POST /identity/v3/projects => generated 301 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 04:59:23.915408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:23.997626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 17/35] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:23.997626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:24.029777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622148d-7591-4b1d-bc14-f00dc28ae99f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.049693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:24.054147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:24.075020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.075952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.076467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.076961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.078197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.079874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.101633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 19/36] 10.222.0.30 () {60 vars in 1131 bytes} [Fri Aug 5 04:59:23 2022] POST /identity/v3/roles => generated 225 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:24.102660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:24.207723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef58dfe-6279-401a-a8b6-cddb511bd27d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.231207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655092c8-2276-45ce-93c9-474c1ec3d1ba None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.263805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4fa7d-0a1d-4b1d-b240-1ff0ebbcdbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.264903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4fa7d-0a1d-4b1d-b240-1ff0ebbcdbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.265377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4fa7d-0a1d-4b1d-b240-1ff0ebbcdbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.265850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4fa7d-0a1d-4b1d-b240-1ff0ebbcdbe9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.267458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4fa7d-0a1d-4b1d-b240-1ff0ebbcdbe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.269316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4fa7d-0a1d-4b1d-b240-1ff0ebbcdbe9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.270720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07894290-193b-4902-9775-bbe9c9d40fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.271365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07894290-193b-4902-9775-bbe9c9d40fed None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.271993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07894290-193b-4902-9775-bbe9c9d40fed None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.271993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07894290-193b-4902-9775-bbe9c9d40fed None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.272621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 18/37] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:24.272833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07894290-193b-4902-9775-bbe9c9d40fed None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.274064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07894290-193b-4902-9775-bbe9c9d40fed None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.274425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a994-9e2b-45dc-829c-ff388b69f57b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.274944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a994-9e2b-45dc-829c-ff388b69f57b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.275408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a994-9e2b-45dc-829c-ff388b69f57b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.282593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 19/38] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:24.283976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8a994-9e2b-45dc-829c-ff388b69f57b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:24.300341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 20/39] 10.222.0.30 () {60 vars in 1131 bytes} [Fri Aug 5 04:59:24 2022] POST /identity/v3/roles => generated 219 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:24.302903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.303211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.303899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.329716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:24.333661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:24.369906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:24.437312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:24.439617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 21/40] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:24.450250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8a994-9e2b-45dc-829c-ff388b69f57b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.473594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:24.489870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f79b3-1c14-4ffb-8a04-bb148826d212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.490581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f79b3-1c14-4ffb-8a04-bb148826d212 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.490860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f79b3-1c14-4ffb-8a04-bb148826d212 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.491140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f79b3-1c14-4ffb-8a04-bb148826d212 None admin] PATH_INFO: `/v3/domains/default/users/c8b08b7805764c45aae169c453594d3d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.494240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f79b3-1c14-4ffb-8a04-bb148826d212 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c8b08b7805764c45aae169c453594d3d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.496511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f79b3-1c14-4ffb-8a04-bb148826d212 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.597292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04939ec-97a1-4c72-b3b0-1ffaca10d736 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.617486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 20/41] 10.222.0.30 () {58 vars in 1361 bytes} [Fri Aug 5 04:59:24 2022] PUT /identity/v3/domains/default/users/c8b08b7805764c45aae169c453594d3d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:24.619224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bcc11-f9a5-47c1-b437-94baa91c8513 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.619520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bcc11-f9a5-47c1-b437-94baa91c8513 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.619820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bcc11-f9a5-47c1-b437-94baa91c8513 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.624283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 21/42] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:24.630177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.630463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.630751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.633347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.634064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.634326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.634608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.635860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.637415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.640730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 22/43] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:24.646367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:24.650794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:24.652575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:24.679927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:24.743881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 22/44] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:24.756780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:24.773205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5217-56a9-4f1b-9c3a-c9c4785b93a7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.811141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebaa42-617b-4cfa-9141-5f1ab598a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.811924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebaa42-617b-4cfa-9141-5f1ab598a838 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.812221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebaa42-617b-4cfa-9141-5f1ab598a838 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.812606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebaa42-617b-4cfa-9141-5f1ab598a838 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.813312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebaa42-617b-4cfa-9141-5f1ab598a838 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.814777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebaa42-617b-4cfa-9141-5f1ab598a838 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.847858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 23/45] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 04:59:24 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:24.848071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:24.868956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54791242-2a50-428c-b77c-cce5b85f759f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:24.907302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246667de-1c02-4749-9d9c-746b00d65082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:24.907981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246667de-1c02-4749-9d9c-746b00d65082 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:24.908267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246667de-1c02-4749-9d9c-746b00d65082 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:24.908542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246667de-1c02-4749-9d9c-746b00d65082 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:24.909289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246667de-1c02-4749-9d9c-746b00d65082 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:24.910843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246667de-1c02-4749-9d9c-746b00d65082 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:24.931757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 23/46] 10.222.0.30 () {60 vars in 1131 bytes} [Fri Aug 5 04:59:24 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 04:59:26.596496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e00f-f08e-4444-a95a-56428ea9325f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:26.597041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e00f-f08e-4444-a95a-56428ea9325f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:26.597041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e00f-f08e-4444-a95a-56428ea9325f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:26.598406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 24/47] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:26.603578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53942098-f6d9-4e0d-9742-255f2bdd2c74 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:26.604047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53942098-f6d9-4e0d-9742-255f2bdd2c74 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:26.604143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53942098-f6d9-4e0d-9742-255f2bdd2c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:26.622610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-53942098-f6d9-4e0d-9742-255f2bdd2c74 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:26.623794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-53942098-f6d9-4e0d-9742-255f2bdd2c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:26.691040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 24/48] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:26.713470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785e00f-f08e-4444-a95a-56428ea9325f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:26.794284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:26.819071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785e00f-f08e-4444-a95a-56428ea9325f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:26.835899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9b72a-21c9-4f0d-98f1-ba1159e3e2a7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:26.836192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9b72a-21c9-4f0d-98f1-ba1159e3e2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:26.836446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9b72a-21c9-4f0d-98f1-ba1159e3e2a7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:26.837836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 25/49] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:26.842709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:26.842992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:26.843266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:26.853342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:26.853937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:26.854220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:26.854502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:26.861625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:26.862814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:26.865056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:26.871125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:26.872096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:26.872857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 25/50] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:26.878196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:26.934841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 26/51] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:26.956954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:26.959903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:26.979651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06537e5d-08b5-49ca-8e3b-bd3afc829a64 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:27.016409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:27.017050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:27.017355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:27.017626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:27.018350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:27.018662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:27.020082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:27.025353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:27.029145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 26/52] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:27.034633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:27.056387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e44e3-c263-4734-951c-db3cf5e51a1c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:27.089578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:27.090281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:27.090596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:27.090877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:27.092064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:27.093552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:27.095125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 27/53] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:27.100625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:27.135005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c94d8-551a-4913-970f-83e875210e35 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:27.168334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:27.168970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:27.169251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:27.169519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:27.169616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:27.170729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:27.172193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:27.173792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 27/54] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:27 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:27.179776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:27.199381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9457cde-07db-4418-af24-51e13b24b356 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:27.231480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:27.232144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:27.232527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:27.232799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:27.233478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:27.234940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:27.243310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:27.264763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 28/55] 10.222.0.30 () {60 vars in 1140 bytes} [Fri Aug 5 04:59:27 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:27.278111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3c8b4-a236-4e74-8638-02a4d35008c5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:27.314853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33beb280-e8a8-4981-b8ee-f4659996644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:27.315531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33beb280-e8a8-4981-b8ee-f4659996644e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:27.315847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33beb280-e8a8-4981-b8ee-f4659996644e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:27.316127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33beb280-e8a8-4981-b8ee-f4659996644e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:27.317381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33beb280-e8a8-4981-b8ee-f4659996644e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:27.318891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33beb280-e8a8-4981-b8ee-f4659996644e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:27.320677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 28/56] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:27.326877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:27.433979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db87e8-81ba-48b1-8613-bad85536b5db None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:27.461760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:27.473293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:27.473936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:27.474217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:27.474489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:27.475282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.domain.id=default, user.id=c8b08b7805764c45aae169c453594d3d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:27.475591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:27.477095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2c205-2994-4aca-a08c-30ea02fea6e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:27.485085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 29/57] 10.222.0.30 () {56 vars in 1322 bytes} [Fri Aug 5 04:59:27 2022] GET /identity/v3/role_assignments?user.id=c8b08b7805764c45aae169c453594d3d&scope.domain.id=default&role.id=47aca6aac2934a5cb816872abfc94994 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:29.273273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1d87c-68fc-45a2-80c2-488cccda3291 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:29.274053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1d87c-68fc-45a2-80c2-488cccda3291 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:29.274053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1d87c-68fc-45a2-80c2-488cccda3291 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:29.276312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 29/58] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:29.281676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:29.282090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:29.282210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:29.308973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:29.310161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:29.380677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 30/59] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:29.405559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1d87c-68fc-45a2-80c2-488cccda3291 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:29.516843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1d87c-68fc-45a2-80c2-488cccda3291 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:29.553786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72ab2d-b1ed-4471-9f51-e3840b5cdf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:29.554705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72ab2d-b1ed-4471-9f51-e3840b5cdf7c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:29.555154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72ab2d-b1ed-4471-9f51-e3840b5cdf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:29.555581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72ab2d-b1ed-4471-9f51-e3840b5cdf7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:29.557733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72ab2d-b1ed-4471-9f51-e3840b5cdf7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:29.560224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72ab2d-b1ed-4471-9f51-e3840b5cdf7c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:29.562821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 30/60] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:29.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:29.677317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb669f-f484-4232-8a46-9b9eac6dcf77 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:29.695052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:29.713706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409e0dc-6b76-4217-91af-c4d1ffd8b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:29.714338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409e0dc-6b76-4217-91af-c4d1ffd8b9a4 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:29.714623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409e0dc-6b76-4217-91af-c4d1ffd8b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:29.714899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409e0dc-6b76-4217-91af-c4d1ffd8b9a4 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:29.715595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409e0dc-6b76-4217-91af-c4d1ffd8b9a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:29.717226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409e0dc-6b76-4217-91af-c4d1ffd8b9a4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:29.746723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 31/61] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:29 2022] POST /identity/v3/users => generated 275 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:32.823366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371556-227c-4725-bf90-5a216c79f2c1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:32.823897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371556-227c-4725-bf90-5a216c79f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:32.824004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371556-227c-4725-bf90-5a216c79f2c1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:32.825299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:32.841454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:32.842495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:32.842960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:32.863958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:32.865159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:32.885955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c1e59-bb64-485b-87b4-04da99ccd62e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:32.886549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c1e59-bb64-485b-87b4-04da99ccd62e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:32.887017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c1e59-bb64-485b-87b4-04da99ccd62e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:32.888740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 32/63] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:32.894795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9236f61-20b6-466b-a3f0-d3dbef9d7538 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:32.895306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9236f61-20b6-466b-a3f0-d3dbef9d7538 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:32.896058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9236f61-20b6-466b-a3f0-d3dbef9d7538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:32.908575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:32.930218 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c9236f61-20b6-466b-a3f0-d3dbef9d7538 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:32.931422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c9236f61-20b6-466b-a3f0-d3dbef9d7538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:32.970131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 32/64] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:33.032067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.042219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 33/65] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:33.088092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810941e-d2d9-4683-8587-17cb27a281af None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.088606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810941e-d2d9-4683-8587-17cb27a281af None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.089031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810941e-d2d9-4683-8587-17cb27a281af None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.090560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 34/66] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:33.095615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.096099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.096539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.121640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:33.123495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:33.176600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a93de6-0e27-433f-b3e0-c5ec36bc3884 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.181096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.215787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 35/67] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:33.215787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.239401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.240735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.241433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.241883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.259714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:33.268480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:33.269609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:33.270579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:33.273277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 33/68] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:33.278144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.295452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.336359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a3e77-699c-48df-92a6-4d1af46b5faf None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.372695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.373466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.373754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.374035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.383607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64229c02-a2cd-4443-b1a8-d1cb24bb80f2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.385147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:33.390193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:33.391023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:33.391752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 36/69] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:33.392400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.425028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ce2c-f878-4daa-bb18-096cd42fa031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.425834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ce2c-f878-4daa-bb18-096cd42fa031 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.426125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ce2c-f878-4daa-bb18-096cd42fa031 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.426407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ce2c-f878-4daa-bb18-096cd42fa031 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.442801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593ce2c-f878-4daa-bb18-096cd42fa031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7593ce2c-f878-4daa-bb18-096cd42fa031 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:33.448246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:33.449084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:33.449832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 34/70] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:33.451302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc36a-2df6-49df-af9f-aaafeeddb41a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.451302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc36a-2df6-49df-af9f-aaafeeddb41a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.451302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc36a-2df6-49df-af9f-aaafeeddb41a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.452676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 35/71] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:33.457415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc36a-2df6-49df-af9f-aaafeeddb41a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.486220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.507700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ccb2e-3e67-47c9-8bde-fd59fd877bdd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.532603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.544396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.545035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.545359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.545639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.546380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:33.546690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:33.548130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:33.557465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 37/72] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:33.562961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.570723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc36a-2df6-49df-af9f-aaafeeddb41a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.605138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.605773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.606059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.606334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.607061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:33.607390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:33.608856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9bdfd-6d01-4aec-a8ec-b43f9c1f7db1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:33.620937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 36/73] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:33.622132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.622480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.622755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.639913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:33.641116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:33.663509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eccab-b1a3-48c7-8d5e-4d41a1d62b6c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.696924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97773794-0871-408e-9407-5f87f8401035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.697556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97773794-0871-408e-9407-5f87f8401035 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.697840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97773794-0871-408e-9407-5f87f8401035 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.698103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97773794-0871-408e-9407-5f87f8401035 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.698855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97773794-0871-408e-9407-5f87f8401035 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:33.699170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97773794-0871-408e-9407-5f87f8401035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:33.700656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97773794-0871-408e-9407-5f87f8401035 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:33.706364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 37/74] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:33.707182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.713805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 38/75] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:33.715276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aaba6-f4b9-45c8-a5e9-9ade39bde83e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.715567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aaba6-f4b9-45c8-a5e9-9ade39bde83e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.715882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aaba6-f4b9-45c8-a5e9-9ade39bde83e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.717328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 39/76] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:33.720029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2aaba6-f4b9-45c8-a5e9-9ade39bde83e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.729979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.771055 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.810620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57834-c692-4d79-a5ee-ccf9dfa64000 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.822764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2aaba6-f4b9-45c8-a5e9-9ade39bde83e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.846479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e34030-170c-4e47-9104-3b4f0947d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.847137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e34030-170c-4e47-9104-3b4f0947d562 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.847424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e34030-170c-4e47-9104-3b4f0947d562 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.847703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e34030-170c-4e47-9104-3b4f0947d562 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.856793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:33.857451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.857814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.858095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.859259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:33.860764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:33.862369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 40/77] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:33.862555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e34030-170c-4e47-9104-3b4f0947d562 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:33.863119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.864005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e34030-170c-4e47-9104-3b4f0947d562 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:33.865662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 38/78] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:33.871308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:33.871597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:33.871901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:33.889113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:33.890260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:33.956041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 39/79] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:33.957221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:33.960200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:33.976532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd395-d02c-4a51-9af6-bd46656dffeb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:33.992932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.013793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.014568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.014848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.015154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.016415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.017887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.019495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 41/80] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:34.024739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.057256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63a51-1201-40f7-9121-f867946c47ba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.091162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.091846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.092160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.092449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.102246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:34.107179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:34.108122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:34.108855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 40/81] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:34.109459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.124970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c6b8-144a-442a-895c-d62bc3fc3d93 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.157771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.158423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.158716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.159010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.172027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.209937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76a6ae-c6bf-4a2a-ad73-d0b081aa066c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.210799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.226290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.227781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.244312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.244990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.245276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.245553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.311200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.312744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.323599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 42/82] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:34.324457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.375886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.404735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.414125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 41/83] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:34.414954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.422374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95426ccb-21f9-471c-95e2-eb3cda094be9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.454898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.455570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.455851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.456125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.457310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.458756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.460399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 43/84] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:34.461119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.514314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff58f8-3477-4992-83e8-17f383bf007a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.548609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.549250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.549537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.549815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.558806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722477a-e9a7-42f1-a2ef-cdc7e3ba9797 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.579238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.593870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.594491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.594772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.595051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.595793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:34.596102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.597502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.605432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 44/85] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:34.606302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.616686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.618173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.621778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.703739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3da6d5-58d1-457b-8673-c43522234a29 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.714928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 42/86] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:34.715808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.737184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.737834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.738121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.738398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.739581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.741082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.742952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 45/87] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:34.743494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.782406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.820785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdce01-99c6-4017-af26-a36b86f95ff7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.822867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:34.840657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d636a9-1cd4-4415-a7e5-a4817f166c6b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:34.853414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.854069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.854351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.854628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.855810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.857303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.859008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 43/88] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:34.859778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.878559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:34.879196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:34.879463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:34.879740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:34.880477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:34.881917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:34.909105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 46/89] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:34 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:34.909978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:34.980805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da05a-2317-494a-b57a-bbba355a8c32 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.020508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:35.023048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68613d8c-789d-424c-9d9b-6af56fe093cc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.027710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.028422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.028721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.028989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.030311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.032509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.034941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 44/90] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:35.036666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:35.067014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.068127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.068410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.068683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.069931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.071383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.073472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 47/91] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:34 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:35.074315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:35.084807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:35.154382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862e83-0616-48d7-a158-fe33942e02a1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.193548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa83d-92ae-4bf6-b138-91bc5a786b17 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.208443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.209143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.209425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.209874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.210784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=c8b08b7805764c45aae169c453594d3d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:35.211096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.212606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.219880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 45/92] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:35 2022] GET /identity/v3/role_assignments?user.id=c8b08b7805764c45aae169c453594d3d&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:35.223168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:35.233881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.234604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.234950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.235289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.236202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=8d8d9ef9df2d4b99998380797a17d038) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:35.236594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.238362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.245999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 48/93] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:35 2022] GET /identity/v3/role_assignments?user.id=8d8d9ef9df2d4b99998380797a17d038&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=3f8debab12324e0da238b80499b5eafe => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:35.246867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:35.305222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:35.350754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:35.375814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae262d4-7e3d-41b2-9dac-a59cb53b364d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.425360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3105d75-10a0-4fc2-9cd0-126e322c9a73 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.456441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.457163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.457445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.457716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.517570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.518149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.518442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.518726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.519727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] RBAC: Adding query filter params (role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=8d8d9ef9df2d4b99998380797a17d038) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:35.520080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.521595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.534618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 49/94] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:35 2022] GET /identity/v3/role_assignments?user.id=8d8d9ef9df2d4b99998380797a17d038&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:35.540482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.542168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.612015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:35.656107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 46/95] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:35.661630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:35.819960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab3cd-4f04-4c01-a09c-748572adda5d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:35.861407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136abed-4d43-452f-a57e-8c50f6ff80be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:35.862077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136abed-4d43-452f-a57e-8c50f6ff80be None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:35.862364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136abed-4d43-452f-a57e-8c50f6ff80be None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:35.862640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136abed-4d43-452f-a57e-8c50f6ff80be None admin] PATH_INFO: `/v3/projects/7e534564c2964d62910d1419aa3847de` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:35.863835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136abed-4d43-452f-a57e-8c50f6ff80be None admin] RBAC: Authorizing `identity:get_project(project_id=7e534564c2964d62910d1419aa3847de)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:35.865309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136abed-4d43-452f-a57e-8c50f6ff80be None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:35.867046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 50/96] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:35 2022] GET /identity/v3/projects/7e534564c2964d62910d1419aa3847de => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:35.873266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:35.996526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116bca1-22ba-4af0-8b47-259c59bbaaae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:36.032293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:36.032927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:36.033214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:36.033488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:36.034279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=7e534564c2964d62910d1419aa3847de, user.id=8d8d9ef9df2d4b99998380797a17d038) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:36.034590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:36.036054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c054c-8ac2-401d-a6af-c3b2d37057b9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:36.041835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 47/97] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:35 2022] GET /identity/v3/role_assignments?user.id=8d8d9ef9df2d4b99998380797a17d038&scope.project.id=7e534564c2964d62910d1419aa3847de&role.id=3f8debab12324e0da238b80499b5eafe => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:37.742543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ac1b-4d34-4a36-8725-1a2252b97a23 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:37.743606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ac1b-4d34-4a36-8725-1a2252b97a23 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:37.744122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ac1b-4d34-4a36-8725-1a2252b97a23 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:37.745686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 51/98] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:37.751033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22d6f-86dc-4e53-980b-6fa821083bf5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:37.751864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22d6f-86dc-4e53-980b-6fa821083bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:37.752361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22d6f-86dc-4e53-980b-6fa821083bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:37.771274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6fd22d6f-86dc-4e53-980b-6fa821083bf5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:37.772792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6fd22d6f-86dc-4e53-980b-6fa821083bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:37.804808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:37.862081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 48/99] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:37.896606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ac1b-4d34-4a36-8725-1a2252b97a23 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:37.899634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:37.911662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66796-fae2-460e-814b-0153e55111e2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:37.911662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66796-fae2-460e-814b-0153e55111e2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:37.911662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66796-fae2-460e-814b-0153e55111e2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:37.911662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 49/100] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:37.917952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc457c-1271-4537-8591-c5abd065982d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:37.918511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc457c-1271-4537-8591-c5abd065982d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:37.918967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc457c-1271-4537-8591-c5abd065982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:37.950341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d0dc457c-1271-4537-8591-c5abd065982d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:37.953157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d0dc457c-1271-4537-8591-c5abd065982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:38.081958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 50/101] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:38.083624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773d54e-faa6-4b8a-97b5-32518e08221e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.084469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ac1b-4d34-4a36-8725-1a2252b97a23 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.086265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773d54e-faa6-4b8a-97b5-32518e08221e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.086729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773d54e-faa6-4b8a-97b5-32518e08221e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.088227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 51/102] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:38.095660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.096214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.096673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.126312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:38.128801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:38.149774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.150848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.151318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.151841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.168191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:38.174530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:38.175444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:38.176208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:38.179318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 52/103] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:38.180472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.250057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 52/104] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:38.251005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.271677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:38.281012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:38.325383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc601a-1843-400f-974e-0274b929f9f5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.362646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123a296-ad04-430f-9d96-883e70e2ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.363328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123a296-ad04-430f-9d96-883e70e2ae5b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.363614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123a296-ad04-430f-9d96-883e70e2ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.363925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123a296-ad04-430f-9d96-883e70e2ae5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.374746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123a296-ad04-430f-9d96-883e70e2ae5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0123a296-ad04-430f-9d96-883e70e2ae5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:38.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:38.381295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:38.382051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:38.386591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 53/105] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:38.387705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2c450-dc64-4e21-a25e-3fa105c77edc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.387977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625c77-1b2e-4965-afad-9f36bf929480 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.388269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625c77-1b2e-4965-afad-9f36bf929480 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.388552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625c77-1b2e-4965-afad-9f36bf929480 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.390325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 54/106] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:38.391065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e625c77-1b2e-4965-afad-9f36bf929480 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.432102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.433368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.434690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.435159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.436125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:38.436623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:38.438204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:38.449831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 53/107] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:38.450992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.503660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e625c77-1b2e-4965-afad-9f36bf929480 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.527214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:38.540984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef91adc-6d0f-4d27-959c-690b4693d982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.541692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef91adc-6d0f-4d27-959c-690b4693d982 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.541972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef91adc-6d0f-4d27-959c-690b4693d982 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.542244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef91adc-6d0f-4d27-959c-690b4693d982 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.543422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef91adc-6d0f-4d27-959c-690b4693d982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:38.545361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef91adc-6d0f-4d27-959c-690b4693d982 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:38.547390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 55/108] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:38.552428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:38.553058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.553709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.553984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.574706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:38.575475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3e75b-a982-4a5e-8128-9829b521bbdc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.576018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:38.618247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.619229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.619723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.620232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.621527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:38.622031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:38.623652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:38.632492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 54/109] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:38.633639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.654495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 56/110] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:38.655606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.746684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aa02a-531e-422a-9f47-4f614617283a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.767187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1903d-fdd5-4d6e-b8da-e51b71b1e446 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.782527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb167af-14d8-4871-81f2-b95ad5e98304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.783186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb167af-14d8-4871-81f2-b95ad5e98304 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.783474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb167af-14d8-4871-81f2-b95ad5e98304 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.783756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb167af-14d8-4871-81f2-b95ad5e98304 None admin] PATH_INFO: `/v3/users/df7458a6ecea442aa3a2058d810f6efd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.788549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:38.796026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:38.799276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb167af-14d8-4871-81f2-b95ad5e98304 None admin] RBAC: Authorizing `identity:get_user(user_id=df7458a6ecea442aa3a2058d810f6efd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:38.800767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb167af-14d8-4871-81f2-b95ad5e98304 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:38.802931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 55/111] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/users/df7458a6ecea442aa3a2058d810f6efd => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:38.803878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.804570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.804732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0decc-c5f6-4099-af3a-3de828998905 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.804879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.804964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0decc-c5f6-4099-af3a-3de828998905 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.805147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.805246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0decc-c5f6-4099-af3a-3de828998905 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.805795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:38.806458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 56/112] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:38.807235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:38.811358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0decc-c5f6-4099-af3a-3de828998905 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.843980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 57/113] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 04:59:38 2022] POST /identity/v3/projects => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:38.846275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:38.940381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0decc-c5f6-4099-af3a-3de828998905 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.977234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae58b0b-d64d-4e02-bc75-b23abe1d9756 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:38.979949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581d764-6081-45b7-9888-867b250c843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:38.980934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581d764-6081-45b7-9888-867b250c843d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:38.981465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581d764-6081-45b7-9888-867b250c843d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:38.981942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581d764-6081-45b7-9888-867b250c843d None admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:38.983494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581d764-6081-45b7-9888-867b250c843d None admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:38.985212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581d764-6081-45b7-9888-867b250c843d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:38.987277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 57/114] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:38.988479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581d764-6081-45b7-9888-867b250c843d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.032898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9e96-8df5-4acc-adaf-9ae75c09e834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.033489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9e96-8df5-4acc-adaf-9ae75c09e834 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.033775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9e96-8df5-4acc-adaf-9ae75c09e834 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.034054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9e96-8df5-4acc-adaf-9ae75c09e834 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.045903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f9e96-8df5-4acc-adaf-9ae75c09e834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6f3f9e96-8df5-4acc-adaf-9ae75c09e834 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:39.051325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:39.052187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:39.052936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 58/115] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:39.054543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.054543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.054543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.065080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.082581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:39.084430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:39.110941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581d764-6081-45b7-9888-867b250c843d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.119804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.148919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde815b-57d0-4bdd-8b8d-de158d59c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.149602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde815b-57d0-4bdd-8b8d-de158d59c987 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.149888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde815b-57d0-4bdd-8b8d-de158d59c987 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.150167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde815b-57d0-4bdd-8b8d-de158d59c987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.162747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 59/116] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:39.163575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.223357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde815b-57d0-4bdd-8b8d-de158d59c987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.225288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde815b-57d0-4bdd-8b8d-de158d59c987 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.263361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf18eb-b852-41e2-951e-ae3c9c9e70d4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.296959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.297534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.297813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.298071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.298179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.299264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.300878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.302626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 60/117] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:39.307863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.334287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.336460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 58/118] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:39.337789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f25d0-af9c-4eb2-9bf9-8401c3843518 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.338119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f25d0-af9c-4eb2-9bf9-8401c3843518 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.338385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f25d0-af9c-4eb2-9bf9-8401c3843518 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.339820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 59/119] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:39.340368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f25d0-af9c-4eb2-9bf9-8401c3843518 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.407742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3be384-0250-4312-b53b-66e8ceeb4cc7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.442023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.442650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.442929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.443205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.443936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:39.444242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.444331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f25d0-af9c-4eb2-9bf9-8401c3843518 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.445609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.454451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 61/120] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:39.459563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.479243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.479540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.480041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.490118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:39.495034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:39.496117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:39.496892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 60/121] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:39.497505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.528812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.562692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.572411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3282-969a-4c3f-b70e-de18be528275 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.604828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086628fa-a1f2-43d7-a28f-52f6a37bb003 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.609472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc7a0a-c856-45f5-bf68-c44a7d8c3615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.610426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc7a0a-c856-45f5-bf68-c44a7d8c3615 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.610902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc7a0a-c856-45f5-bf68-c44a7d8c3615 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.611370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc7a0a-c856-45f5-bf68-c44a7d8c3615 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/c8b08b7805764c45aae169c453594d3d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.613591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc7a0a-c856-45f5-bf68-c44a7d8c3615 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ad666675744c88976e6b45e1415a3a, user_id=c8b08b7805764c45aae169c453594d3d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.615362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc7a0a-c856-45f5-bf68-c44a7d8c3615 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.632438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 62/122] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:39 2022] PUT /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/c8b08b7805764c45aae169c453594d3d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:39.633968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.634259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.634537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.648469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.649538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.650050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.650512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.652113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.653991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:39.654288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.655514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:39.656686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 61/123] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:39.657754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.747186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 63/124] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:39.753978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.779624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.790238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46d6ea-5b89-42c3-8c04-d63df836cc9a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.838526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.839118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.839544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.839856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.841359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.842881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.844695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 62/125] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:39.845290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.865959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:39.890025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96826eb-0ea4-46ea-a5ba-5ec2327a28d8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:39.932352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:39.932992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:39.933273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:39.933548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:39.934753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:39.935073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:39.936794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:39.947112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 64/126] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:39.948548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:39.979744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2bc8-7d06-4685-8cf7-3808b8e919ba None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.034773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.036599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.037100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.037787 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.039041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] RBAC: Adding query filter params (role.id=f1bde6b54454477cb807392e68475a8c, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=df7458a6ecea442aa3a2058d810f6efd) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:40.039529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.041397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.050576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 63/127] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/role_assignments?user.id=df7458a6ecea442aa3a2058d810f6efd&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=f1bde6b54454477cb807392e68475a8c => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:40.052751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.086186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8f795-0780-48d1-a83d-f1059f1240f0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.098423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:40.141489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.142396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.142681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.144672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.167488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:40.181607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:40.182529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:40.183276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:40.184379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 65/128] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:40.187344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.198793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:40.243083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a1b88-e840-4485-a377-8c3bb3867f42 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.284147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.284792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.285082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.285362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.286824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.290064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.292552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 64/129] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:40 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:40.293217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.361063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c369e8b-df4a-48f2-8b75-848742852b5a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.400177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:40.404199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eac06-e6df-4b02-b15a-240ad38e65fd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.441851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.442499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.442786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.443067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.443809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:40.444141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.446024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.455023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 65/130] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:40.460598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.480521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.481211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.481532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.481782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.485353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.486949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.491799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 66/131] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:40 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:40.491799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.574460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d232cb-8ca7-4dec-8f35-47f220ea9fa4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.641272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:40.642295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.643054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.643347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.643631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/8d8d9ef9df2d4b99998380797a17d038/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.649459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ad666675744c88976e6b45e1415a3a, user_id=8d8d9ef9df2d4b99998380797a17d038, role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.650764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.679025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6598c05-9a97-4ea1-8b55-9a011d1c53b6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.689431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 66/132] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:40 2022] PUT /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/8d8d9ef9df2d4b99998380797a17d038/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:40.690075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.723158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.724299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.724800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.725260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.726846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.728875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.736348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 67/133] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:40 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:40.738514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.753404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:40.894745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f951d-838e-4eaf-aa8c-7311d3bf0603 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:40.932889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:40.933791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:40.934210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:40.934648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] PATH_INFO: `/v3/projects/7e534564c2964d62910d1419aa3847de` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:40.936087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] RBAC: Authorizing `identity:get_project(project_id=7e534564c2964d62910d1419aa3847de)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:40.938271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:40.940175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 68/134] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:40 2022] GET /identity/v3/projects/7e534564c2964d62910d1419aa3847de => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:40.949002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:40.962666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:41.021751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150586d3-dfa9-4233-98c1-a907d7321c7a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:41.118511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552829-fa38-4f44-8f43-56efb5105bbc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:41.120487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:41.121162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.121450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.121729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.122948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:41.124573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:41.126251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 67/135] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:40 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:41.132229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:41.176499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9121-4f42-435a-8fc5-e2a464fecad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:41.177208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9121-4f42-435a-8fc5-e2a464fecad0 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.177509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9121-4f42-435a-8fc5-e2a464fecad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.177800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9121-4f42-435a-8fc5-e2a464fecad0 None admin] PATH_INFO: `/v3/projects/7e534564c2964d62910d1419aa3847de/users/8d8d9ef9df2d4b99998380797a17d038/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.180697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9121-4f42-435a-8fc5-e2a464fecad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e534564c2964d62910d1419aa3847de, user_id=8d8d9ef9df2d4b99998380797a17d038, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:41.182208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9121-4f42-435a-8fc5-e2a464fecad0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:41.212479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 69/136] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:40 2022] PUT /identity/v3/projects/7e534564c2964d62910d1419aa3847de/users/8d8d9ef9df2d4b99998380797a17d038/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:41.213827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e909d-f353-48fe-99f5-4b0dfafae6b8 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.214132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e909d-f353-48fe-99f5-4b0dfafae6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.214412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e909d-f353-48fe-99f5-4b0dfafae6b8 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.215999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 70/137] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:41.223013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.223324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.223607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.278779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:41.280380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:41.348158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:41.375409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5641-93a5-4d0e-922a-c89f1ea3aca6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:41.444518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:41.453397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 71/138] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:41.462638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:41.463370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.463652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.463957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/8d8d9ef9df2d4b99998380797a17d038/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.469323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ad666675744c88976e6b45e1415a3a, user_id=8d8d9ef9df2d4b99998380797a17d038, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:41.470839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:41.481895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:41.503988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 68/139] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:41 2022] PUT /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/8d8d9ef9df2d4b99998380797a17d038/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:41.607286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67b138-b639-4f83-95f5-e2259bf2c021 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:41.649338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cee48b-8a16-49e3-ab47-2648c9e30dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:41.650612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cee48b-8a16-49e3-ab47-2648c9e30dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.651092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cee48b-8a16-49e3-ab47-2648c9e30dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.651549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cee48b-8a16-49e3-ab47-2648c9e30dd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.653130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cee48b-8a16-49e3-ab47-2648c9e30dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:41.655044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cee48b-8a16-49e3-ab47-2648c9e30dd4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:41.657242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 72/140] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:41.665160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:41.787387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c48518-13ee-457d-822c-4823dfd404fb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:41.827469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4598c07-6958-41ce-b393-66790d191815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:41.828511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4598c07-6958-41ce-b393-66790d191815 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:41.828983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4598c07-6958-41ce-b393-66790d191815 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:41.829442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4598c07-6958-41ce-b393-66790d191815 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:41.830361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4598c07-6958-41ce-b393-66790d191815 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:41.832055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4598c07-6958-41ce-b393-66790d191815 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:41.866029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 69/141] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:41 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:42.205270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb541-48e9-4d15-a307-03318379cc84 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:42.206318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb541-48e9-4d15-a307-03318379cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:42.206796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb541-48e9-4d15-a307-03318379cc84 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:42.208681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 73/142] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:42.225316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:42.226231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:42.226706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:42.239227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:42.253095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:42.255400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:42.327449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 70/143] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:42.350511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb541-48e9-4d15-a307-03318379cc84 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:42.436433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:42.477511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb541-48e9-4d15-a307-03318379cc84 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:42.520316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:42.520976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:42.521261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:42.521541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:42.536630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:42.541606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:42.542470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:42.543285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 74/144] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:42.552111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:42.686450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3bb5f-0cb6-4062-8112-178b162ac51b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:42.724146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:42.728527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:42.729053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:42.729605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:42.729890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:42.730629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:42.730935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:42.732365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:42.742789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 71/145] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:42.749202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:42.865345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511df92-f0d8-4729-8901-b3caef54dbcc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:42.894419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:42.903470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:42.904171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:42.904721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:42.905069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] PATH_INFO: `/v3/users/c8b08b7805764c45aae169c453594d3d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:42.906569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] RBAC: Authorizing `identity:get_user(user_id=c8b08b7805764c45aae169c453594d3d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:42.908045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:42.909666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 75/146] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:42 2022] GET /identity/v3/users/c8b08b7805764c45aae169c453594d3d => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:42.915362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:43.020728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807d4ce-cfc7-4b2f-9265-2b43d628471f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:43.054970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e27a8e-805b-4c3f-92a7-2d9bc8f43304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:43.055611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e27a8e-805b-4c3f-92a7-2d9bc8f43304 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.056798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e27a8e-805b-4c3f-92a7-2d9bc8f43304 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.057074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e27a8e-805b-4c3f-92a7-2d9bc8f43304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.118697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:43.126003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e27a8e-805b-4c3f-92a7-2d9bc8f43304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:43.127877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e27a8e-805b-4c3f-92a7-2d9bc8f43304 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:43.229285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 72/147] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:43.235104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:43.347299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eb78c-0f26-4d68-babd-362e3aa8fbba None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:43.389535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82091-86f3-4f67-93d8-e66578039634 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.389856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82091-86f3-4f67-93d8-e66578039634 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.390102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82091-86f3-4f67-93d8-e66578039634 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.391466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 73/148] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:43.396576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0b971-3e54-4b21-81b6-9542b07dd72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:43.397247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b971-3e54-4b21-81b6-9542b07dd72a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.397557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b971-3e54-4b21-81b6-9542b07dd72a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.397840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b971-3e54-4b21-81b6-9542b07dd72a None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.399015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b971-3e54-4b21-81b6-9542b07dd72a None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:43.400527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b971-3e54-4b21-81b6-9542b07dd72a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:43.402338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 76/149] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:43 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:43.408284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82091-86f3-4f67-93d8-e66578039634 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:43.421191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.421482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.421744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.471529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:43.472764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:43.573680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:43.577004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:43.588118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82091-86f3-4f67-93d8-e66578039634 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:43.598260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 77/150] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:43.619038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:43.619807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.620178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.620553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.621456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=c8b08b7805764c45aae169c453594d3d) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:43.621858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:43.623476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd61c9-8d74-4a47-9578-e687fe823aa6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:43.630119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 74/151] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:43 2022] GET /identity/v3/role_assignments?user.id=c8b08b7805764c45aae169c453594d3d&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:43.630895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:43.765537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab915a6f-9d91-4883-bd67-276505c1d084 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:43.797845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05714fa-5944-4556-9a8b-9d8b6e68b259 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.798177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05714fa-5944-4556-9a8b-9d8b6e68b259 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.798401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05714fa-5944-4556-9a8b-9d8b6e68b259 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.800064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 75/152] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:43.807881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.808228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.808325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:43.808524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.809381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:43.809883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:43.810361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:43.823178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:43.828930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:43.829805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:43.830576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:43.832955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 78/153] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:43.839145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:43.840430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:43.840661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:43.928627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:43.928949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 76/154] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:43.965437 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:43.985592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79864-3620-4560-a6dd-ffa0a802d40c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.024302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.025419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.026003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.026471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.027396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:44.027920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.029612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:44.039929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 79/155] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:44.074546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.109708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:44.127034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545006ec-cf70-4abe-b809-79c24dcae47a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.182318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.182918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.183202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.183480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.205434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:44.210383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:44.211392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:44.212344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:44.215989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 77/156] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 04:59:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:44.217655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.218564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7abad8-f1da-4511-bb5c-944d4ee32981 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.235927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:44.260372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.261392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.261928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.262396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] PATH_INFO: `/v3/users/df7458a6ecea442aa3a2058d810f6efd` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.263932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] RBAC: Authorizing `identity:get_user(user_id=df7458a6ecea442aa3a2058d810f6efd)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.265639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:44.267516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 80/157] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:44 2022] GET /identity/v3/users/df7458a6ecea442aa3a2058d810f6efd => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:44.273437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.348302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cd52-536d-4adc-be1e-c279cde149e3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.386179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.387071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.387494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.387959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.388868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:44.389364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.390918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:44.400271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 78/158] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 04:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:44.407031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.413026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1f9b6-ad8a-4756-9787-1a89057174ae None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.422429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:44.456287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.457329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.457842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.458387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.459819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.461705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:44.463621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 81/159] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:44 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:44.469949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.517212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8badb91-8ff7-4fab-8981-42ae4488d174 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.525453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:44.561144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.561799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.562082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.562356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.563632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.565203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:44.566843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 79/160] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:44 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:44.572746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.584032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716ac54-0487-4555-9ba5-04febcb12936 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.620128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c27cfc-dcfa-4a6a-9b55-38110c70987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.621271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c27cfc-dcfa-4a6a-9b55-38110c70987b None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.621749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c27cfc-dcfa-4a6a-9b55-38110c70987b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.622228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c27cfc-dcfa-4a6a-9b55-38110c70987b None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/df7458a6ecea442aa3a2058d810f6efd/roles/f1bde6b54454477cb807392e68475a8c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.624695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c27cfc-dcfa-4a6a-9b55-38110c70987b None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ad666675744c88976e6b45e1415a3a, user_id=df7458a6ecea442aa3a2058d810f6efd, role_id=f1bde6b54454477cb807392e68475a8c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.626387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c27cfc-dcfa-4a6a-9b55-38110c70987b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:44.643341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 82/161] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:44 2022] PUT /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a/users/df7458a6ecea442aa3a2058d810f6efd/roles/f1bde6b54454477cb807392e68475a8c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 04:59:44.647842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a3fe8-2ce8-484d-96e3-cb7671aef92f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.648416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a3fe8-2ce8-484d-96e3-cb7671aef92f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.648891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a3fe8-2ce8-484d-96e3-cb7671aef92f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.650746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 83/162] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:44.656875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.657556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.658063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.698067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:44.698067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:44.706074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:44.758295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3e824-1a46-43f8-b851-a5e8ce92c4d5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:44.840012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afbc02-5459-405e-878e-bc0814bac47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:44.841592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afbc02-5459-405e-878e-bc0814bac47b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:44.842347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afbc02-5459-405e-878e-bc0814bac47b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:44.843008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afbc02-5459-405e-878e-bc0814bac47b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:44.910386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 84/163] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:44.953401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afbc02-5459-405e-878e-bc0814bac47b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:44.953590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:44.955372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afbc02-5459-405e-878e-bc0814bac47b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:45.044380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:45.116537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:45.144615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bdc71-aeff-4be2-aefe-50a41f48056a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:45.156882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 80/164] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:45.158322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ca8ce-512e-410c-9173-75f8046761fd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.158426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ca8ce-512e-410c-9173-75f8046761fd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.158628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ca8ce-512e-410c-9173-75f8046761fd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.162137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 81/165] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:45.166812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ca8ce-512e-410c-9173-75f8046761fd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:45.209056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5bb1d-4078-4583-8ab7-151fa40f13fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:45.209707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5bb1d-4078-4583-8ab7-151fa40f13fd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.209983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5bb1d-4078-4583-8ab7-151fa40f13fd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.210256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5bb1d-4078-4583-8ab7-151fa40f13fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.230080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5bb1d-4078-4583-8ab7-151fa40f13fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-84b5bb1d-4078-4583-8ab7-151fa40f13fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:45.235087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:45.235963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:45.237330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 85/166] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:45.238046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.239745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.241163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.270721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:45.271933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:45.338131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ca8ce-512e-410c-9173-75f8046761fd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:45.394498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 86/167] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:45.395353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:45.408178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:45.408838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.409117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.409389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.410668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:45.412243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:45.414074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 82/168] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:45 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:45.419972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:45.455597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:45.507974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:45.586476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455b285-4769-48b0-a37a-abf010a83e8c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:45.616828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115afe-f663-469f-b791-871fa7aa69f5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:45.653136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:45.653775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.654052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.654399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.655124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:45.655431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:45.656854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:45.667445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 87/169] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:45.668287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:45.676134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:45.677052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.677547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.678424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.679526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] RBAC: Adding query filter params (role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=8d8d9ef9df2d4b99998380797a17d038) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:45.680076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:45.681749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:45.688499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 83/170] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:45 2022] GET /identity/v3/role_assignments?user.id=8d8d9ef9df2d4b99998380797a17d038&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:45.692883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:45.770104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261f94b-4834-48f5-b17c-8e7aa3e688f2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:45.780212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:45.808878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:45.809697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:45.809984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:45.810261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:45.822881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:45.829221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:45.830068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:45.830838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 88/171] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:45.836946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:45.925895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17a967-3734-4fa5-ad20-ba0ab75a50a2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:45.954147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:45.988106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75dd7a-d459-4997-8f82-59272dc44165 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:46.044530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.045260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.045541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.045813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.046551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:46.046860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.048291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.049715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.050397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.050686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.051008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.052253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.054128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.056001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 84/172] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:45 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:46.057618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 89/173] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:46.066502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.066681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.092248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:46.183303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d15556-16e1-4073-85b4-1e2ee3beb3c7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:46.223325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.224028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.224323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.224602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] PATH_INFO: `/v3/users/8d8d9ef9df2d4b99998380797a17d038` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.225985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8d9ef9df2d4b99998380797a17d038)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.227837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.229659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 90/174] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/users/8d8d9ef9df2d4b99998380797a17d038 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:46.235201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.344953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae84e6-9d75-480c-861e-cadb7739074b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:46.394859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:46.416531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f91f6-f6a1-42b1-ac48-403d392b8796 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:46.426813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.427456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.428361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.428784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.455592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fddb4-2494-49a4-9969-3c530fc8b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.456504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fddb4-2494-49a4-9969-3c530fc8b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.457361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fddb4-2494-49a4-9969-3c530fc8b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.457793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fddb4-2494-49a4-9969-3c530fc8b0a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.520909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:46.541412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fddb4-2494-49a4-9969-3c530fc8b0a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.543207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fddb4-2494-49a4-9969-3c530fc8b0a9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.553772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.555499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.630420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:46.651129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 91/175] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 04:59:46.652815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75b711-1c22-47c1-97d3-f578b1a5da64 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.653363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75b711-1c22-47c1-97d3-f578b1a5da64 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.653778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75b711-1c22-47c1-97d3-f578b1a5da64 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.655238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 92/176] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:46.660407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 85/177] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:46.660587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085f28b-0a20-4aae-9bb7-30f2698b3c34 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.660819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085f28b-0a20-4aae-9bb7-30f2698b3c34 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.661108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085f28b-0a20-4aae-9bb7-30f2698b3c34 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.661950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.662524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 93/178] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:46.663541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b629f-6828-4395-a47c-067491489813 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.663931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b629f-6828-4395-a47c-067491489813 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.664213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b629f-6828-4395-a47c-067491489813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.683377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c01b629f-6828-4395-a47c-067491489813 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:46.684833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c01b629f-6828-4395-a47c-067491489813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:46.745587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:46.765922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 94/179] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:46.767201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b629f-6828-4395-a47c-067491489813 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.770970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6beec-aa0c-41f1-a71b-d5b3a78d328c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:46.807072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d855b-0994-4e07-aa5a-2d04bafd67d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.808127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d855b-0994-4e07-aa5a-2d04bafd67d2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.808605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d855b-0994-4e07-aa5a-2d04bafd67d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.809094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d855b-0994-4e07-aa5a-2d04bafd67d2 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.810587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d855b-0994-4e07-aa5a-2d04bafd67d2 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.812294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d855b-0994-4e07-aa5a-2d04bafd67d2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.814213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 86/180] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:46.815731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.816240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.816696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.835008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:46.836574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:46.879336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b629f-6828-4395-a47c-067491489813 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:46.895395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:46.905158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 87/181] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:46.906375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.942069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:46.943115 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:46.943609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:46.944111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] PATH_INFO: `/v3/projects/7e534564c2964d62910d1419aa3847de` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:46.945501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] RBAC: Authorizing `identity:get_project(project_id=7e534564c2964d62910d1419aa3847de)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:46.947143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:46.949084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 95/182] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/projects/7e534564c2964d62910d1419aa3847de => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:46.950029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:46.984970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.009331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7587ccf-4aeb-44e7-ad7e-7b1ca0071813 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.043369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.044265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.044742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.045212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.056148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a23d2-e32f-43e0-b08b-9ff6b847e452 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.056740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:47.061962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:47.063044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:47.064084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:47.066073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 88/183] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:47.067030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.095503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.096431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.096805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.097288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.098145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=8d8d9ef9df2d4b99998380797a17d038) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:47.098551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.100236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.106725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 96/184] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:46 2022] GET /identity/v3/role_assignments?user.id=8d8d9ef9df2d4b99998380797a17d038&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=3f8debab12324e0da238b80499b5eafe => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:47.107664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.159578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.174553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff21c0-700a-4e23-9eee-a50a8b0e6656 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.211435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.212417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.212834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.213242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.216774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.225729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e8fb0-dbc2-4fda-8bd7-97a1554defdf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.227138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:47.232312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:47.233978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 89/185] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:47.236012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.281448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.282433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.282862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.283269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.284363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=7e534564c2964d62910d1419aa3847de, user.id=8d8d9ef9df2d4b99998380797a17d038) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:47.284849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.286584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.293867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 97/186] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/role_assignments?user.id=8d8d9ef9df2d4b99998380797a17d038&scope.project.id=7e534564c2964d62910d1419aa3847de&role.id=3f8debab12324e0da238b80499b5eafe => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:47.295049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.351813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1682ddc-3466-4a54-a7c1-859978c644dd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.388848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.389912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.390401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.390864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.391854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:47.392398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.394074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.403814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 90/187] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:47.409264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.419482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0f8a0-87e0-4c13-b935-e9b50c8ffc57 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.442826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.484965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.485613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.485896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.486169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.487011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:47.487324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.488832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a0983-ceeb-4879-971c-407d1621aabe None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.493860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.501838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 98/188] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:47.503144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a19546-c464-4d85-ad3e-16a22fd08605 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.503431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a19546-c464-4d85-ad3e-16a22fd08605 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.503796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a19546-c464-4d85-ad3e-16a22fd08605 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.506225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 99/189] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:47.511272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a19546-c464-4d85-ad3e-16a22fd08605 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.550703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44b6d-b7fc-48ca-803b-6a7ed2adc587 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.594293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9da376-391c-4c39-864a-d6fce2fbe024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.595325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da376-391c-4c39-864a-d6fce2fbe024 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.595877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da376-391c-4c39-864a-d6fce2fbe024 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.596158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da376-391c-4c39-864a-d6fce2fbe024 None admin] PATH_INFO: `/v3/users/28574a4730ec476a91f6089fcd362854` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.611798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9da376-391c-4c39-864a-d6fce2fbe024 None admin] RBAC: Authorizing `identity:get_user(user_id=28574a4730ec476a91f6089fcd362854)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.612111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9da376-391c-4c39-864a-d6fce2fbe024 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.613917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 91/190] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/users/28574a4730ec476a91f6089fcd362854 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:47.616093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.616382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.616637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.634638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:47.636799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:47.652245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a19546-c464-4d85-ad3e-16a22fd08605 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.698645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.700036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.700664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.700912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] PATH_INFO: `/v3/users/28574a4730ec476a91f6089fcd362854` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.702621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] RBAC: Authorizing `identity:get_user(user_id=28574a4730ec476a91f6089fcd362854)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.706070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.706070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 100/191] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/users/28574a4730ec476a91f6089fcd362854 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:47.706622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.722224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 92/192] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:47.722929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:47.760239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.775943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.825594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d50e1-bff4-4177-bb90-6fbe976be397 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.831988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8c7c1-9e81-445a-8b16-58750bcc848b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:47.858579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.859209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.859488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.859758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.865385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e319f-8609-40db-be2d-1ce3ea32d11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:47.866021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e319f-8609-40db-be2d-1ce3ea32d11d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:47.866300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e319f-8609-40db-be2d-1ce3ea32d11d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:47.866558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e319f-8609-40db-be2d-1ce3ea32d11d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:47.924433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.925826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.932379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e319f-8609-40db-be2d-1ce3ea32d11d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:47.933835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e319f-8609-40db-be2d-1ce3ea32d11d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:47.963040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:47.976606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.023945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 101/193] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:48.024795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.031660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 93/194] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:48.033407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9817b-1bc5-4c1e-9477-f63aff4f64df None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.033695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9817b-1bc5-4c1e-9477-f63aff4f64df None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.033968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9817b-1bc5-4c1e-9477-f63aff4f64df None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.035698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 94/195] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:48.040444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9817b-1bc5-4c1e-9477-f63aff4f64df None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.122735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefc605-f3d3-45e9-90c0-e7c80be63d0b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.143005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9817b-1bc5-4c1e-9477-f63aff4f64df None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.156512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.157166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.157450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.157726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.158880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.160401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.162005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 102/196] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:48.166856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.171459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.177339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371afd1-9c21-423a-bc22-eec9117b0336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.178349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371afd1-9c21-423a-bc22-eec9117b0336 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.178627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371afd1-9c21-423a-bc22-eec9117b0336 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.178897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371afd1-9c21-423a-bc22-eec9117b0336 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.180069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371afd1-9c21-423a-bc22-eec9117b0336 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.181533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371afd1-9c21-423a-bc22-eec9117b0336 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.183193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 95/197] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:48.184424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.184707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.184978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.203237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:48.204504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:48.214284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.264347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bce905-b23c-4524-b6df-39ff9cc54c40 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.271373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 96/198] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:48.272205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.297189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.297932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.298187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.298464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.299657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.301150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.302818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 103/199] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:48.303944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.377526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.383212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec7b6e-2f3f-4a15-8245-c06b80b00f02 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.402215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea2a4-0d4d-42df-bbbf-00a68139f7af None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.417292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.417930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.418219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.418500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.419190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.420731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.435203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.435991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.436307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.436598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.437450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.437598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=28574a4730ec476a91f6089fcd362854) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:48.437784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.439367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.452367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 97/200] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:48 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:48.452367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.463490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 104/201] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/role_assignments?user.id=28574a4730ec476a91f6089fcd362854&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:48.463490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.591557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fff9de-7ea9-43bd-a290-2036f2cb0c63 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.611333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c82d9-ddf7-4d0a-8183-374c4401b3e9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.637061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.637678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.637943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.638211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.652041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.652392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:48.652681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.652962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.653240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.654039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] RBAC: Adding query filter params (role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=28574a4730ec476a91f6089fcd362854) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:48.654422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.655916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:48.658153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:48.659097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:48.659910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:48.661207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 98/202] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:48.663108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 105/203] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/role_assignments?user.id=28574a4730ec476a91f6089fcd362854&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:48.665796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.736416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.786090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06704a0-3a18-434e-ac72-eaa826eb408a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.824538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.825193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.825467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.825746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.826602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:48.826909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:48.829783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:48.838775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 99/204] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:48.844592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:48.847561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:48.953051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbe690-452d-4e15-ba66-68a55e9fffb6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:48.995245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:48.996212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:48.996630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:48.997030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] PATH_INFO: `/v3/users/df7458a6ecea442aa3a2058d810f6efd` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:48.998409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] RBAC: Authorizing `identity:get_user(user_id=df7458a6ecea442aa3a2058d810f6efd)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:49.000059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:49.002170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 106/205] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:48 2022] GET /identity/v3/users/df7458a6ecea442aa3a2058d810f6efd => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:49.008498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:49.156784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe73aa7-f66d-4878-9290-b39eba168adc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:49.189797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:49.202477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:49.203167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:49.203394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:49.203667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:49.280621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:49.282099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:49.390347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 100/206] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:49.396175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:49.491834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:49.524570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb412e0-91d1-4b81-8285-e6255e89fde7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:49.569397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec56e7-b395-4800-adfd-436ce8fdd8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:49.570068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec56e7-b395-4800-adfd-436ce8fdd8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:49.570348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec56e7-b395-4800-adfd-436ce8fdd8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:49.570619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec56e7-b395-4800-adfd-436ce8fdd8f8 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:49.571828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec56e7-b395-4800-adfd-436ce8fdd8f8 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:49.573303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec56e7-b395-4800-adfd-436ce8fdd8f8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:49.575296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 107/207] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:49 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:49.581071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:49.622678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:49.730719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f9c26-ffcc-47dd-9773-1a74556bbf6d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:49.765220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:49.765738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:49.765949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:49.766184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:49.766974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] RBAC: Adding query filter params (role.id=f1bde6b54454477cb807392e68475a8c, scope.project.id=c9ad666675744c88976e6b45e1415a3a, user.id=df7458a6ecea442aa3a2058d810f6efd) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:49.767284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:49.768836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cf200-ef82-44d4-b479-ec6f13ee3d59 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:49.778817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 101/208] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:49 2022] GET /identity/v3/role_assignments?user.id=df7458a6ecea442aa3a2058d810f6efd&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=f1bde6b54454477cb807392e68475a8c => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:50.869258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827569a0-71b6-48fd-8243-a13bad32abda None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:50.869783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827569a0-71b6-48fd-8243-a13bad32abda None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:50.869783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827569a0-71b6-48fd-8243-a13bad32abda None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:50.871385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 108/209] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:50.876939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1a1c-91fe-4e9f-85d9-677652a6d863 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:50.877339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1a1c-91fe-4e9f-85d9-677652a6d863 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:50.877504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1a1c-91fe-4e9f-85d9-677652a6d863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:50.888704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06eaa0-7189-4e59-a842-138aed3e8439 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:50.889000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06eaa0-7189-4e59-a842-138aed3e8439 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:50.889267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06eaa0-7189-4e59-a842-138aed3e8439 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:50.890634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 109/210] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:50.895668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:50.895990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:50.896276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:50.896734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b36b1a1c-91fe-4e9f-85d9-677652a6d863 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:50.897918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b36b1a1c-91fe-4e9f-85d9-677652a6d863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:50.914546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:50.915751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:50.972340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 102/211] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:50.974330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9238ab-95c6-4f1d-945f-3c77b26de468 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:50.974626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9238ab-95c6-4f1d-945f-3c77b26de468 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:50.974904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9238ab-95c6-4f1d-945f-3c77b26de468 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:50.976213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 103/212] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:50.981525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:50.981817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:50.982102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:50.987091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 110/213] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:50.996556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.005443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:51.006750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:51.024976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.071716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.082538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 104/214] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:51.083374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.105990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7de91-eb88-43d3-a185-77c3c5cbe10d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.138190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f22929-948d-4a0a-a624-863a063feee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.138856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22929-948d-4a0a-a624-863a063feee9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.139132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22929-948d-4a0a-a624-863a063feee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.139403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22929-948d-4a0a-a624-863a063feee9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.149033 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f22929-948d-4a0a-a624-863a063feee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e0f22929-948d-4a0a-a624-863a063feee9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:51.153887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:51.154864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:51.155826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 111/215] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 04:59:51.157008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0e8ac-e5c5-4d71-b9ca-9c38f9c1f274 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.157294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0e8ac-e5c5-4d71-b9ca-9c38f9c1f274 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.157549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0e8ac-e5c5-4d71-b9ca-9c38f9c1f274 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.158876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 112/216] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:51.160557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0e8ac-e5c5-4d71-b9ca-9c38f9c1f274 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.187953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd275-1234-479a-907c-3b7870b4ee63 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.221316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.221969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.222254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.222531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.232983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:51.237940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:51.238886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:51.239676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 105/217] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:51.240427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.253070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.262011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0e8ac-e5c5-4d71-b9ca-9c38f9c1f274 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.287448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.295854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b14599-4795-45e3-8658-9f3e535a5a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.296504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b14599-4795-45e3-8658-9f3e535a5a13 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.296784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b14599-4795-45e3-8658-9f3e535a5a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.297052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b14599-4795-45e3-8658-9f3e535a5a13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.308220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b14599-4795-45e3-8658-9f3e535a5a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-25b14599-4795-45e3-8658-9f3e535a5a13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:51.313212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:51.314127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:51.314896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 113/218] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:51.315529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.315852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.316048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.333150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:51.334328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:51.346492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699a7d-8eaf-476b-a03a-5266e334790d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.380147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.380763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.381046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.381322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.382048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:51.382359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.383756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.391739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 106/219] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.392557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.404331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 114/220] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:51.415967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.457231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.490235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e771f-af5b-4641-ac47-39dec23b4f49 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.514270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5a5f7-8f8a-458c-a6a1-90131c5188ef None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.522798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.523424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.523704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.524004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.524718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:51.525021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.526415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.531214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.534166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 107/221] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.534941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.554006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.554640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.554923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.555202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.555965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:51.556292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.557678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.565695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 115/222] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.566328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.637078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd80e-fec5-4c01-a2fe-9da7f96d8ae2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.664529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559b8a8-de6c-4aac-9355-6aac90f0f0af None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.667291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.674366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.675054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.675345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.675619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] PATH_INFO: `/v3/users/28574a4730ec476a91f6089fcd362854` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.676849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] RBAC: Authorizing `identity:get_user(user_id=28574a4730ec476a91f6089fcd362854)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.678682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.680551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 108/223] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/users/28574a4730ec476a91f6089fcd362854 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.681120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.698260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.698912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.699197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.699521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.700836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.702492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.704159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 116/224] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.704901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.752916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.780316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10232dbf-d911-4437-9a8f-36754ae83608 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.803718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f25608-4725-4852-bb87-26a5a4bbb016 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.813981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.814637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.814934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.815207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] PATH_INFO: `/v3/users/28574a4730ec476a91f6089fcd362854` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.816398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] RBAC: Authorizing `identity:get_user(user_id=28574a4730ec476a91f6089fcd362854)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.817857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.819729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 109/225] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/users/28574a4730ec476a91f6089fcd362854 => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.820401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.837591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.838251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.838540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.838824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] PATH_INFO: `/v3/users/d48b1f05765e42269303401bd6e75362` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.851737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] RBAC: Authorizing `identity:get_user(user_id=d48b1f05765e42269303401bd6e75362)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.853244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.854873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 117/226] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/users/d48b1f05765e42269303401bd6e75362 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.855611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.884520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.919714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045602e7-9ddb-47e8-9f76-bf57ce675f02 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.954526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716dd7c-241b-4fc9-8c34-2d19509fc80f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:51.955041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.955784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.956077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.956344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.957517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.958973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:51.960930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 110/227] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:51.961685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:51.963063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:51.991525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:51.992175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:51.992449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:51.992718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:51.993442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:51.994887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.018346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 118/228] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:51 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:52.019021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:52.075589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce678872-3f2b-47f0-9064-999233f30d2e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:52.114421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:52.115079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:52.115361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:52.115634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:52.117144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:52.118428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.120277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 111/229] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:51 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:52.120862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:52.123860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:52.140916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e249f4-c8a2-4bcf-8d97-3bb47f3fa4a9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:52.182552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:52.183456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:52.183740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:52.184044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:52.219950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:52.240105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f8614-94eb-4d74-b262-6623906f861b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:52.268837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:52.270318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.282786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:52.283559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:52.283898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:52.284176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/28574a4730ec476a91f6089fcd362854/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:52.287692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c4224b99e4c15856ab6c5e76cd861, user_id=28574a4730ec476a91f6089fcd362854, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:52.289276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.306783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 112/230] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:52 2022] PUT /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/28574a4730ec476a91f6089fcd362854/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:52.312619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:52.388636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:52.405770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 119/231] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:52.412205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:52.435435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120166c-b1c0-4647-afd9-524c0b772b9d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:52.490803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:52.491952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:52.492550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:52.492972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/28574a4730ec476a91f6089fcd362854/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:52.495967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c4224b99e4c15856ab6c5e76cd861, user_id=28574a4730ec476a91f6089fcd362854, role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:52.497795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.503664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:52.538126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 113/232] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:52 2022] PUT /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/28574a4730ec476a91f6089fcd362854/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:52.551212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a412c-7ceb-44d9-896d-dc5842339fbb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:52.592423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d2f4d-ecea-40c0-b88d-3022f41cea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:52.593299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2f4d-ecea-40c0-b88d-3022f41cea3d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:52.593668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2f4d-ecea-40c0-b88d-3022f41cea3d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:52.594031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2f4d-ecea-40c0-b88d-3022f41cea3d None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:52.595385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d2f4d-ecea-40c0-b88d-3022f41cea3d None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:52.597062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d2f4d-ecea-40c0-b88d-3022f41cea3d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.598945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 120/233] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:52 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:52.605359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:52.736508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8f5f7-c13c-47b8-b7eb-c7991ee455ae None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:52.763887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:52.778575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:52.779889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:52.780586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:52.781238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:52.782105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=d48b1f05765e42269303401bd6e75362) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:52.782482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:52.784032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463ec67-5a7f-4e20-b96e-a9f1c2a57526 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:52.789941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 114/234] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:52 2022] GET /identity/v3/role_assignments?user.id=d48b1f05765e42269303401bd6e75362&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=3f8debab12324e0da238b80499b5eafe => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:54.402167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb63625-6097-402e-ba1a-3cea50ffd419 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.402686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb63625-6097-402e-ba1a-3cea50ffd419 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.402686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb63625-6097-402e-ba1a-3cea50ffd419 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.404447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 121/235] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:54.410360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a650376-a635-484e-93a4-9b3a6e0952f0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.410845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a650376-a635-484e-93a4-9b3a6e0952f0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.410951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a650376-a635-484e-93a4-9b3a6e0952f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.429648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1a650376-a635-484e-93a4-9b3a6e0952f0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:54.430841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a650376-a635-484e-93a4-9b3a6e0952f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:54.526053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 115/236] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:54.548629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb63625-6097-402e-ba1a-3cea50ffd419 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:54.661440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb63625-6097-402e-ba1a-3cea50ffd419 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:54.691952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e24a4-5352-44b1-bf12-0a4f50bbf7d9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.692254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e24a4-5352-44b1-bf12-0a4f50bbf7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.692510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e24a4-5352-44b1-bf12-0a4f50bbf7d9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.694027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 116/237] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:54.695481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:54.700096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a5f3f-d62a-4764-9fa0-afbf0d152638 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.700387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a5f3f-d62a-4764-9fa0-afbf0d152638 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.700664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a5f3f-d62a-4764-9fa0-afbf0d152638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.709355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:54.714368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.714653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.714933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.725008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f85a5f3f-d62a-4764-9fa0-afbf0d152638 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:54.726503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f85a5f3f-d62a-4764-9fa0-afbf0d152638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:54.738592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:54.743540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:54.744427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:54.745221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:54.745816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 122/238] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:54.753317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:54.871217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 117/239] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:54.881156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc4a75-0e68-4f20-be48-97ffad95bb8c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:54.916000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:54.916607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.917209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.917528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.918255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:54.918564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:54.919287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db0505-56f7-4ff5-b8e1-01e276476e1d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.919668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db0505-56f7-4ff5-b8e1-01e276476e1d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.920045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833d107-ee4e-461b-ad05-b1ec4502a80a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:54.920210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db0505-56f7-4ff5-b8e1-01e276476e1d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.925613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 118/240] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:54.926387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db0505-56f7-4ff5-b8e1-01e276476e1d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:54.929588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 123/241] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:54.931074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:54.931446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:54.931724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:54.954178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:54.955650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:54.981317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.019518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.047454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 124/242] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:55.048452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.067726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db0505-56f7-4ff5-b8e1-01e276476e1d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.102619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.103274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.103558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.103860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.105105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.106608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.108362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 119/243] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:55.109126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.170380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b77c19-e424-4069-99f2-f5258e57ae38 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.204049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.204733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.205015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.205294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] PATH_INFO: `/v3/users/6f962f3b9c7e4ea2a9d2967a12c95829` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.208586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68952bf0-9cc8-4c3d-8406-ba91ef557ed6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.219419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] RBAC: Authorizing `identity:get_user(user_id=6f962f3b9c7e4ea2a9d2967a12c95829)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.220926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.227057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 125/244] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/users/6f962f3b9c7e4ea2a9d2967a12c95829 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:55.227057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.229231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.251227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ef4a1-bcba-4f39-ba97-659719f3912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.253084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ef4a1-bcba-4f39-ba97-659719f3912b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.253364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ef4a1-bcba-4f39-ba97-659719f3912b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.253634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ef4a1-bcba-4f39-ba97-659719f3912b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.263699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ef4a1-bcba-4f39-ba97-659719f3912b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:55.263840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-231ef4a1-bcba-4f39-ba97-659719f3912b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:55.268695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:55.269576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:55.270334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 120/245] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:55.271453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffc846-d762-4630-975d-29d67bf63fe1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.271453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffc846-d762-4630-975d-29d67bf63fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.271595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffc846-d762-4630-975d-29d67bf63fe1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.275890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 121/246] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:55.275890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffc846-d762-4630-975d-29d67bf63fe1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.332124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd515c3-bdc3-46f4-84c6-8f41a52b4963 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.365983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.366614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.366898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.367162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.367881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.369323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.383195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffc846-d762-4630-975d-29d67bf63fe1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.391748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 126/247] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:55 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 04:59:55.392506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.435634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff36a1-c91f-415e-b1a4-d55e9d4de975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.436208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff36a1-c91f-415e-b1a4-d55e9d4de975 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.436495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff36a1-c91f-415e-b1a4-d55e9d4de975 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.436767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff36a1-c91f-415e-b1a4-d55e9d4de975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.489338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.508644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.515520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb13f01-3ccc-432f-a42f-197aac4bc3d5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.523070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff36a1-c91f-415e-b1a4-d55e9d4de975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.524615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff36a1-c91f-415e-b1a4-d55e9d4de975 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.556399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.557758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.557758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.557758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.558279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:55.558583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.560010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacd2a3-0cbf-46b0-96f2-204d34edbb86 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.571007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 127/248] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:55.571007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.571007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.571007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.588970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:55.590154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:55.641672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 122/249] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:55.643104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516d6f3-b149-490e-a00d-8cee9ed9c3bc None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.643396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516d6f3-b149-490e-a00d-8cee9ed9c3bc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.643671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516d6f3-b149-490e-a00d-8cee9ed9c3bc None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.645094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 123/250] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:55.648824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516d6f3-b149-490e-a00d-8cee9ed9c3bc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.672406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 128/251] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:55.673422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.744565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.779780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516d6f3-b149-490e-a00d-8cee9ed9c3bc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.782525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:55.796248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243e50-cbf7-44d8-8f02-6d642d39a9fb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:55.823073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18a40e-1c64-4165-a5ac-ce5247357a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.823936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a40e-1c64-4165-a5ac-ce5247357a7d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.824376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a40e-1c64-4165-a5ac-ce5247357a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.824653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a40e-1c64-4165-a5ac-ce5247357a7d None admin] PATH_INFO: `/v3/users/28574a4730ec476a91f6089fcd362854` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.826473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18a40e-1c64-4165-a5ac-ce5247357a7d None admin] RBAC: Authorizing `identity:get_user(user_id=28574a4730ec476a91f6089fcd362854)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.828411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18a40e-1c64-4165-a5ac-ce5247357a7d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.830244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 124/252] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/users/28574a4730ec476a91f6089fcd362854 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:55.832012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.832293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.832616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.842851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:55.843532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:55.843841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:55.844239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:55.845429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:55.846987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:55.849359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 129/253] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:55.850376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.853950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:55.856208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:55.934683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 125/254] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:55.935680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:55.988783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8369d-c78d-4e36-8f9f-7f32f837557e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.030313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.031134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.031505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.032001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.037237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.044972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:56.050445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:56.051441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 04:59:56.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:56.053230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 130/255] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:56.053892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.074005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.075539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d286a-a1a3-4b38-8732-e8cc3f2ffa5d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.120007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.121576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.122066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.122528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.165360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ddd40-d42d-4288-a4d3-5961e478dd3f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.200878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.201865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.202365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.202643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.203586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] RBAC: Adding query filter params (role.id=f1bde6b54454477cb807392e68475a8c, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=6f962f3b9c7e4ea2a9d2967a12c95829) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:56.203942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:56.205402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:56.210843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:56.212625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:56.219886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 131/256] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/role_assignments?user.id=6f962f3b9c7e4ea2a9d2967a12c95829&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=f1bde6b54454477cb807392e68475a8c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:56.220955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.296836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.336179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 126/257] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:56.337439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.430902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.455128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a2515-766e-4f99-80c0-deaa7345590a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.495103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a13ca-a317-4462-917f-90a098cca6b2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.508137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.508807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.509303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.509375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.533478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.534800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.535295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.535754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.536834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:56.537332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:56.538921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:56.540643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:56.546257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:56.547191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:56.548006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 132/258] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:56.551194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 127/259] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:56.552371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.553722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.637916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.693677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845cffd-6a97-4492-b863-5606b70f3ffe None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.740527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.741195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.741479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.741757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.744485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:56.745946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:56.751788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 133/260] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:56.751788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.776250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.776571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a655b-92cd-4afd-bf00-202f64d3731a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.816994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.818132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:56.818658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:56.819121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:56.820055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:56.820699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:56.822352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:56.831340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 128/261] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:56.832940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:56.955524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4385a-4c31-406b-887f-42c0a8cb005f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:56.980208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:56.995823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:56.997684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e724dda-8617-47bf-802f-98d18d713384 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.003798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.003798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.003798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] PATH_INFO: `/v3/users/28574a4730ec476a91f6089fcd362854` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.003798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] RBAC: Authorizing `identity:get_user(user_id=28574a4730ec476a91f6089fcd362854)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.003798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.006133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 134/262] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/users/28574a4730ec476a91f6089fcd362854 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:57.006710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:57.064722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:57.065368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.065655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.065934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.066749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=28574a4730ec476a91f6089fcd362854) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:57.067130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.068549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.086945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 129/263] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:56 2022] GET /identity/v3/role_assignments?user.id=28574a4730ec476a91f6089fcd362854&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:57.087576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:57.146847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:57.193422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188eb6e-1bf8-4dbf-8edd-4a0db6313473 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.241846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4eda-463e-4e61-82cd-dbd3cb24ff17 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.275811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:57.275811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.275811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.275811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] PATH_INFO: `/v3/users/d48b1f05765e42269303401bd6e75362` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.277144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d48b1f05765e42269303401bd6e75362)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.278691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.280544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 135/264] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:57 2022] GET /identity/v3/users/d48b1f05765e42269303401bd6e75362 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:57.286137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:57.296638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:57.297749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.298274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.298908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.353999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:57.404121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.406041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.534996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 130/265] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 04:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 04:59:57.538542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8d27-00cc-46ea-b304-b432b33690f0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.540554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:57.608661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:57.609258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.609545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.609826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.611259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.612793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.614810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 136/266] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:57 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:57.619977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:57.620209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:57.646600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:57.686727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efe02-171b-4ce9-8fc5-a89b45eac154 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.739472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:57.740163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.740442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.740712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.742808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.744305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.745998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 131/267] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:57 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:57.752926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:57.799490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2337eef-c571-4391-a782-46339ec290f7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.834795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee0da1-e289-4a04-9320-de83677093ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:57.835857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee0da1-e289-4a04-9320-de83677093ab None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.836457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee0da1-e289-4a04-9320-de83677093ab None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.836943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee0da1-e289-4a04-9320-de83677093ab None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/d48b1f05765e42269303401bd6e75362/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.839128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee0da1-e289-4a04-9320-de83677093ab None admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c4224b99e4c15856ab6c5e76cd861, user_id=d48b1f05765e42269303401bd6e75362, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:57.840877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee0da1-e289-4a04-9320-de83677093ab None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:57.855741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 137/268] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:57 2022] PUT /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/d48b1f05765e42269303401bd6e75362/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 04:59:57.857665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffa03c-9f9e-49fd-ba71-b5c729e3bdaa None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.858202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffa03c-9f9e-49fd-ba71-b5c729e3bdaa None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.858771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffa03c-9f9e-49fd-ba71-b5c729e3bdaa None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.864886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 138/269] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:57.867090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed13a6-2df8-40c6-8497-9140b4f469c0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.867598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed13a6-2df8-40c6-8497-9140b4f469c0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.868097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed13a6-2df8-40c6-8497-9140b4f469c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.888654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d0ed13a6-2df8-40c6-8497-9140b4f469c0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:57.890264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d0ed13a6-2df8-40c6-8497-9140b4f469c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:57.913451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01449733-438b-4597-aa7b-627c6ab0edf5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:57.967835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 139/270] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:57.971993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:57.977171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e299e8-7980-4b56-9100-09034d290e1e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.977463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e299e8-7980-4b56-9100-09034d290e1e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.977722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e299e8-7980-4b56-9100-09034d290e1e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.979090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 140/271] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 04:59:57.985310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:57.985632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:57.985861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:57.992309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:58.004672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070690-9910-4561-a844-88d727840e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.005333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070690-9910-4561-a844-88d727840e8f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.005611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070690-9910-4561-a844-88d727840e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.005881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070690-9910-4561-a844-88d727840e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.006686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08070690-9910-4561-a844-88d727840e8f None admin] RBAC: Adding query filter params (role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=28574a4730ec476a91f6089fcd362854) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:58.007001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08070690-9910-4561-a844-88d727840e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:58.008498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08070690-9910-4561-a844-88d727840e8f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:58.018587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:58.020879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:58.025627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 132/272] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 04:59:57 2022] GET /identity/v3/role_assignments?user.id=28574a4730ec476a91f6089fcd362854&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:58.026296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070690-9910-4561-a844-88d727840e8f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.117802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 141/273] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 04:59:58.155098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.180395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070690-9910-4561-a844-88d727840e8f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:58.221220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.222334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.222775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.223221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.235536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:58.240964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:58.241833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 04:59:58.243070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 133/274] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:58.248876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.287058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a8063-b2e4-4f14-8309-5659b2a7b1e3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:58.312503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:58.319592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:58.325986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.327053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.327476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.327931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.330098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:58.331726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:58.338156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 142/275] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 04:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:58.339545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.367230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c71e36-13f3-4b6f-a1b5-21dff9534fee None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:58.404709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.405482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.405840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.406915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.406915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:58.407403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:58.408932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:58.418479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 134/276] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:58.424136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.451047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118e05-e070-40cd-b9bb-bf589b94fc9d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:58.487058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.488005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.488427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.488877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.489745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:58.491377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:58.522170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 143/277] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 04:59:58 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 04:59:58.533953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f489ca-7ca5-47b6-89eb-1c1e052c5ff0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:58.550709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:58.573919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37baa0-18d5-43c9-a4e7-73ebecc5a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.574698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37baa0-18d5-43c9-a4e7-73ebecc5a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.575102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37baa0-18d5-43c9-a4e7-73ebecc5a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.575450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37baa0-18d5-43c9-a4e7-73ebecc5a3a5 None admin] PATH_INFO: `/v3/users/47927500ff8b4391be14cac3d3e1538e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.592311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37baa0-18d5-43c9-a4e7-73ebecc5a3a5 None admin] RBAC: Authorizing `identity:get_user(user_id=47927500ff8b4391be14cac3d3e1538e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:58.593913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37baa0-18d5-43c9-a4e7-73ebecc5a3a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:58.596068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 135/278] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:58 2022] GET /identity/v3/users/47927500ff8b4391be14cac3d3e1538e => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 04:59:58.601951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.680251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:58.738691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3288a5-b37b-4e20-8622-9dc4bfe8cac2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:58.747900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bb341-3fe3-4dcd-86fb-87f2d9501168 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.748321 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bb341-3fe3-4dcd-86fb-87f2d9501168 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.748696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bb341-3fe3-4dcd-86fb-87f2d9501168 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.750322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 136/279] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 04:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 04:59:58.755722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.756144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.756522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.777473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 04:59:58.777697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:58.778151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:58.778428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:58.778700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:58.779248 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 04:59:58.779554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, user.id=47927500ff8b4391be14cac3d3e1538e, scope.system=all) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:58.779902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:58.781407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:58.789819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 144/280] 10.222.0.30 () {56 vars in 1308 bytes} [Fri Aug 5 04:59:58 2022] GET /identity/v3/role_assignments?user.id=47927500ff8b4391be14cac3d3e1538e&scope.system=all&role.id=3f8debab12324e0da238b80499b5eafe => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:58.865193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 137/281] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 04:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 04:59:58.887877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:58.998370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3c21-726c-4739-93ee-cd15e8317fae None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:59.019746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:59.037494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:59.038523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:59.038941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:59.039278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:59.050051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 04:59:59.055945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 04:59:59.057050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 04:59:59.058047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 04:59:59.058793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 145/282] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 04:59:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 04:59:59.064854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:59.191997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372867b-e052-4211-b849-9d3b91a66687 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:59.193935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:59.230317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:59.231045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:59.231388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:59.231724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:59.232577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 04:59:59.232944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:59.234407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:59.247929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 138/283] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 04:59:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:59.249215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:59.367663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50fd07-5ccd-4994-abed-4dd1727494d4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:59.416572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201247-9746-473d-a394-722d5c1217cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:59.417319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201247-9746-473d-a394-722d5c1217cb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:59.417668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201247-9746-473d-a394-722d5c1217cb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:59.418007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201247-9746-473d-a394-722d5c1217cb None admin] PATH_INFO: `/v3/users/6f962f3b9c7e4ea2a9d2967a12c95829` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:59.419272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71201247-9746-473d-a394-722d5c1217cb None admin] RBAC: Authorizing `identity:get_user(user_id=6f962f3b9c7e4ea2a9d2967a12c95829)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:59.420881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71201247-9746-473d-a394-722d5c1217cb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:59.422553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 146/284] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 04:59:59 2022] GET /identity/v3/users/6f962f3b9c7e4ea2a9d2967a12c95829 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:59.428718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:59.543693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84db682-2051-4797-9098-8a2781564d48 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:59.589425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf0a99-dc54-4841-a313-e9393fcd927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:59.590126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf0a99-dc54-4841-a313-e9393fcd927e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:59.590359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf0a99-dc54-4841-a313-e9393fcd927e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:59.590631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf0a99-dc54-4841-a313-e9393fcd927e None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:59.592453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf0a99-dc54-4841-a313-e9393fcd927e None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:59.593928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf0a99-dc54-4841-a313-e9393fcd927e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:59.595938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 139/285] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 04:59:59 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 04:59:59.601569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201247-9746-473d-a394-722d5c1217cb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 04:59:59.710574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 04:59:59.827699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201247-9746-473d-a394-722d5c1217cb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 04:59:59.874340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518140a4-f473-4066-afd7-fcb1616c3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 04:59:59.875031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518140a4-f473-4066-afd7-fcb1616c3a53 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 04:59:59.875318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518140a4-f473-4066-afd7-fcb1616c3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 04:59:59.875595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518140a4-f473-4066-afd7-fcb1616c3a53 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/6f962f3b9c7e4ea2a9d2967a12c95829/roles/f1bde6b54454477cb807392e68475a8c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 04:59:59.877600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518140a4-f473-4066-afd7-fcb1616c3a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c4224b99e4c15856ab6c5e76cd861, user_id=6f962f3b9c7e4ea2a9d2967a12c95829, role_id=f1bde6b54454477cb807392e68475a8c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 04:59:59.879054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518140a4-f473-4066-afd7-fcb1616c3a53 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 04:59:59.906135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 147/286] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 04:59:59 2022] PUT /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/users/6f962f3b9c7e4ea2a9d2967a12c95829/roles/f1bde6b54454477cb807392e68475a8c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:00.165199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e141dfc-f397-4e9f-abff-5e99b90f7e22 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:00.165559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e141dfc-f397-4e9f-abff-5e99b90f7e22 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:00.165761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e141dfc-f397-4e9f-abff-5e99b90f7e22 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:00.167274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 140/287] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:00.172817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:00.174034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:00.174034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:00.196224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:00.197427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:00.265849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 148/288] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:00.301017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e141dfc-f397-4e9f-abff-5e99b90f7e22 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:00.347899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:00.438423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e141dfc-f397-4e9f-abff-5e99b90f7e22 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:00.489323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:00.489977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:00.490263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:00.490535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:00.506849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:00.512304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:00.513145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:00.513957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 141/289] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:00.520923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:00.578322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:00.644204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231d15c-664b-4b5d-9fdc-414fe3b4c000 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:00.678260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:00.678895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:00.679175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:00.679448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:00.680201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:00.680504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:00.681881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:00.690590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 149/290] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:00.696090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:00.906119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:00.937943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180bf03-a309-4867-afdc-a5f0cc2557d3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:01.054171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ddb09-0afb-4578-b976-e04bc5c68f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:01.054828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ddb09-0afb-4578-b976-e04bc5c68f15 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.055116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ddb09-0afb-4578-b976-e04bc5c68f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.055383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ddb09-0afb-4578-b976-e04bc5c68f15 None admin] PATH_INFO: `/v3/users/d48b1f05765e42269303401bd6e75362` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.056599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ddb09-0afb-4578-b976-e04bc5c68f15 None admin] RBAC: Authorizing `identity:get_user(user_id=d48b1f05765e42269303401bd6e75362)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:01.058066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ddb09-0afb-4578-b976-e04bc5c68f15 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:01.059941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 142/291] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:00 2022] GET /identity/v3/users/d48b1f05765e42269303401bd6e75362 => generated 297 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:01.068923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.202694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:01.205622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35017e38-b512-44a9-9f31-e2b5943dc74d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:01.226245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c57bb-83a8-4bbc-97c8-2dd7eb563a8b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.226529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c57bb-83a8-4bbc-97c8-2dd7eb563a8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.226809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c57bb-83a8-4bbc-97c8-2dd7eb563a8b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.228538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 143/292] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:01.233751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.234040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.234351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.241189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f39084-117b-4438-88c3-5dcc9bddbdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:01.242092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f39084-117b-4438-88c3-5dcc9bddbdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.242547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f39084-117b-4438-88c3-5dcc9bddbdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.243016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f39084-117b-4438-88c3-5dcc9bddbdc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.254299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:01.255472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:01.316876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f39084-117b-4438-88c3-5dcc9bddbdc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:01.318635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f39084-117b-4438-88c3-5dcc9bddbdc5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:01.350957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 144/293] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:01.373534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.420836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 150/294] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:01.421619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:01.424430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7a312-61d1-47c4-a03a-d24b5e1cf5c3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.425010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7a312-61d1-47c4-a03a-d24b5e1cf5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.425721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7a312-61d1-47c4-a03a-d24b5e1cf5c3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.427295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 151/295] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:01.428575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7a312-61d1-47c4-a03a-d24b5e1cf5c3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.467012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:01.477055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287e2d-6364-41ce-98e1-d51ba546df02 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:01.515963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b99da2-cfe0-415e-b869-6aa7f6a498b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:01.516580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b99da2-cfe0-415e-b869-6aa7f6a498b8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.517239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b99da2-cfe0-415e-b869-6aa7f6a498b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.517522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b99da2-cfe0-415e-b869-6aa7f6a498b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.519010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b99da2-cfe0-415e-b869-6aa7f6a498b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:01.520472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b99da2-cfe0-415e-b869-6aa7f6a498b8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:01.522271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 145/296] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:01.528237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.528528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.528811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.540958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7a312-61d1-47c4-a03a-d24b5e1cf5c3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:01.546705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:01.547943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:01.578264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acf91e-b5ff-4642-8172-a8980d9190a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:01.578964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acf91e-b5ff-4642-8172-a8980d9190a6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.579251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acf91e-b5ff-4642-8172-a8980d9190a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.579552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acf91e-b5ff-4642-8172-a8980d9190a6 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.580739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acf91e-b5ff-4642-8172-a8980d9190a6 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:01.582183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acf91e-b5ff-4642-8172-a8980d9190a6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:01.588637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 152/297] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:01 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:01.588637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06cb2-c24d-41a2-b69d-30c3c87787ed None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.588637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06cb2-c24d-41a2-b69d-30c3c87787ed None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.588637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06cb2-c24d-41a2-b69d-30c3c87787ed None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.588637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 153/298] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:01.588637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06cb2-c24d-41a2-b69d-30c3c87787ed None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.622118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 146/299] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:01.622998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.660860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:01.696364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06cb2-c24d-41a2-b69d-30c3c87787ed None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:01.717629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:01.731469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e14c5e-5f7a-4df4-9b0c-bd87ab27a675 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:01.732856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a92e0-2e4f-4177-afcf-93a8c7bbae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:01.733487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a92e0-2e4f-4177-afcf-93a8c7bbae8c None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.733770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a92e0-2e4f-4177-afcf-93a8c7bbae8c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.734066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a92e0-2e4f-4177-afcf-93a8c7bbae8c None admin] PATH_INFO: `/v3/groups` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.734734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a92e0-2e4f-4177-afcf-93a8c7bbae8c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:01.736191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a92e0-2e4f-4177-afcf-93a8c7bbae8c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:01.762164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 154/300] 10.222.0.30 () {60 vars in 1134 bytes} [Fri Aug 5 05:00:01 2022] POST /identity/v3/groups => generated 229 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:01.762164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.762164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.762164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.775241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:01.776513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:01.777727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:01.778483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:01.778934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:01.779372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:01.780372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=d48b1f05765e42269303401bd6e75362) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:01.780777 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:01.782384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:01.789760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 147/301] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:01 2022] GET /identity/v3/role_assignments?user.id=d48b1f05765e42269303401bd6e75362&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=3f8debab12324e0da238b80499b5eafe => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:01.797051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.912138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 155/302] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:01.948221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:01.966251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993fffa-0a0b-4d7a-9fc1-d1bbb791e0ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.026556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.027160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.027451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.027731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.044301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:02.046261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:02.051370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:02.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:02.053510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 148/303] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:02.062996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.083361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.106747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064ac4a-57ac-49b7-8118-972ef4663bce None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.151827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.152500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.152783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.153116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.166046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:02.172224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:02.173103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:02.173912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 156/304] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:02.179661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.203898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ad02e-b147-4881-a5fe-af98acce40c1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.247987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.248804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.249089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.249659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.250642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:02.250947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:02.252460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:02.263633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 149/305] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:02.269948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.298453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f51fa6-18f4-482e-909f-36d94656edf1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.320031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.341880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.342803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.343620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.344116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.345003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:02.345489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:02.347857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:02.364367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 157/306] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:02.365490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.373848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.401373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939828cf-04e6-48de-b42c-5e80dc17af5e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.437312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e57ee3-bc50-4dee-9785-54b9d11f4853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.438014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e57ee3-bc50-4dee-9785-54b9d11f4853 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.438301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e57ee3-bc50-4dee-9785-54b9d11f4853 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.438575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e57ee3-bc50-4dee-9785-54b9d11f4853 None admin] PATH_INFO: `/v3/users/47927500ff8b4391be14cac3d3e1538e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.439755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e57ee3-bc50-4dee-9785-54b9d11f4853 None admin] RBAC: Authorizing `identity:get_user(user_id=47927500ff8b4391be14cac3d3e1538e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:02.441333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e57ee3-bc50-4dee-9785-54b9d11f4853 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:02.443099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 150/307] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/users/47927500ff8b4391be14cac3d3e1538e => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:02.448913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506e360-e80b-450a-bb28-79e56475d497 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.449201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506e360-e80b-450a-bb28-79e56475d497 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.449480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506e360-e80b-450a-bb28-79e56475d497 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.450728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 151/308] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:02.451485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506e360-e80b-450a-bb28-79e56475d497 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.475291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd995b-db1a-4166-a838-d61027e3b4cd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.509705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67284c5e-18f4-4f6c-957b-0f2ddf745210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.510431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67284c5e-18f4-4f6c-957b-0f2ddf745210 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.510712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67284c5e-18f4-4f6c-957b-0f2ddf745210 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.510987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67284c5e-18f4-4f6c-957b-0f2ddf745210 None admin] PATH_INFO: `/v3/users/02022474ce36470a9b56d53fb5baa295` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.524846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67284c5e-18f4-4f6c-957b-0f2ddf745210 None admin] RBAC: Authorizing `identity:get_user(user_id=02022474ce36470a9b56d53fb5baa295)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:02.526290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67284c5e-18f4-4f6c-957b-0f2ddf745210 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:02.528158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 158/309] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/users/02022474ce36470a9b56d53fb5baa295 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:02.533809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829a891-2a83-4790-b146-895c41f13066 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.534092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829a891-2a83-4790-b146-895c41f13066 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.534364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829a891-2a83-4790-b146-895c41f13066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.551424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a829a891-2a83-4790-b146-895c41f13066 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:02.552574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a829a891-2a83-4790-b146-895c41f13066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:02.561038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506e360-e80b-450a-bb28-79e56475d497 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.583669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.602533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.603452 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.603560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.603751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] PATH_INFO: `/v3/system/users/47927500ff8b4391be14cac3d3e1538e/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.605226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=47927500ff8b4391be14cac3d3e1538e, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:02.606618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:02.617119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 152/310] 10.222.0.30 () {58 vars in 1334 bytes} [Fri Aug 5 05:00:02 2022] PUT /identity/v3/system/users/47927500ff8b4391be14cac3d3e1538e/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:02.624920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.625156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 159/311] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:02.636868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.648220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829a891-2a83-4790-b146-895c41f13066 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.775368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43311bdb-4827-4be1-90c9-6027f2d89ee9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.775969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829a891-2a83-4790-b146-895c41f13066 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:02.811251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.811853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.812131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.812408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.814067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:02.814839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:02.815085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:02.815361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:02.816286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] RBAC: Adding query filter params (role.id=f1bde6b54454477cb807392e68475a8c, user.id=02022474ce36470a9b56d53fb5baa295, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:02.816643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:02.818103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:02.823664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:02.823826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 153/312] 10.222.0.30 () {56 vars in 1308 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/role_assignments?user.id=02022474ce36470a9b56d53fb5baa295&scope.system=all&role.id=f1bde6b54454477cb807392e68475a8c => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:02.828977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:02.829835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:02.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 160/313] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:02.836558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:02.915347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:02.958280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4a8a-8d94-4afb-9a29-7d57ed4970c0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:03.000119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:03.001039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.001323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.001597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:03.002323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:03.002629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:03.004040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:03.014416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 154/314] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:03.024040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:03.057568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:03.130973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff0ce6-154c-4c3d-85bb-bcc66a00a6ea None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:03.168273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:03.168926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.169205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.169479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] PATH_INFO: `/v3/users/6f962f3b9c7e4ea2a9d2967a12c95829` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:03.170808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] RBAC: Authorizing `identity:get_user(user_id=6f962f3b9c7e4ea2a9d2967a12c95829)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:03.172350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:03.174298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 161/315] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:03 2022] GET /identity/v3/users/6f962f3b9c7e4ea2a9d2967a12c95829 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:03.184904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:03.292550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f506f5b-59f5-48a7-92fd-07c020ada217 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:03.322878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:03.329389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:03.330022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.330307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.330584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:03.402215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:03.403695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:03.504346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 155/316] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:03.512939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:03.610972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:03.617783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f17537-0491-457c-84b9-c5b55e807535 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:03.653597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c698-f74a-4767-ab39-7ca743da392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:03.654614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c698-f74a-4767-ab39-7ca743da392a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.655192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c698-f74a-4767-ab39-7ca743da392a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.655657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c698-f74a-4767-ab39-7ca743da392a None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:03.657082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07c698-f74a-4767-ab39-7ca743da392a None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:03.658730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07c698-f74a-4767-ab39-7ca743da392a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:03.661062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 162/317] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:03 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:03.666678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:03.701104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:03.772230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61a0da-0681-4716-bf20-f0ca7f3c1685 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:03.807025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:03.807820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.808024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.808294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:03.809084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] RBAC: Adding query filter params (role.id=f1bde6b54454477cb807392e68475a8c, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861, user.id=6f962f3b9c7e4ea2a9d2967a12c95829) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:03.809396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:03.810893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342c14a-67b8-4f5f-8e6d-50024471a90a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:03.817403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 156/318] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:03 2022] GET /identity/v3/role_assignments?user.id=6f962f3b9c7e4ea2a9d2967a12c95829&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=f1bde6b54454477cb807392e68475a8c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:03.970005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e8dee-6ebe-45ad-ab81-3f500f42f898 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.970368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e8dee-6ebe-45ad-ab81-3f500f42f898 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.970563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e8dee-6ebe-45ad-ab81-3f500f42f898 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:03.972247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 163/319] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:03.981356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:03.981759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:03.982210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.002437 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:04.003632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:04.073939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 157/320] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:04.096595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e8dee-6ebe-45ad-ab81-3f500f42f898 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:04.199521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e8dee-6ebe-45ad-ab81-3f500f42f898 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:04.233806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cbbfd-bf4b-4149-9912-a42588790072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:04.234419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cbbfd-bf4b-4149-9912-a42588790072 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:04.234699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cbbfd-bf4b-4149-9912-a42588790072 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:04.234969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cbbfd-bf4b-4149-9912-a42588790072 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.236123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cbbfd-bf4b-4149-9912-a42588790072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:04.237554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cbbfd-bf4b-4149-9912-a42588790072 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:04.239555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 164/321] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:04.245648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:04.275379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:04.350788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29652bb-0164-42d6-b6b8-ba9f965fe655 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:04.384384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38007c-a6d1-4f7e-8418-f695beac276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:04.385015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38007c-a6d1-4f7e-8418-f695beac276f None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:04.385293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38007c-a6d1-4f7e-8418-f695beac276f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:04.385560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38007c-a6d1-4f7e-8418-f695beac276f None admin] PATH_INFO: `/v3/groups` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.386217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38007c-a6d1-4f7e-8418-f695beac276f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:04.387670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38007c-a6d1-4f7e-8418-f695beac276f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:04.411801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 158/322] 10.222.0.30 () {60 vars in 1134 bytes} [Fri Aug 5 05:00:04 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:04.774833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407db8c-15da-4a94-95d9-6cb1465425ad None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:04.775202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407db8c-15da-4a94-95d9-6cb1465425ad None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:04.775395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407db8c-15da-4a94-95d9-6cb1465425ad None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.777048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 165/323] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:04.782610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:04.782898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:04.783176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.805823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:04.807047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:04.855564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6d376-0ed9-4d2a-a3bf-4da548bfa20a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:04.855919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6d376-0ed9-4d2a-a3bf-4da548bfa20a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:04.856149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6d376-0ed9-4d2a-a3bf-4da548bfa20a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.857806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 166/324] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:04.863122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:04.863640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:04.864131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:04.879087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 159/325] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:04.885855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:04.887350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:04.903014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:04.920171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:04.923172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:04.977193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 167/326] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:05.004497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.024987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f46eca-ca4f-4e1a-bb00-4021c35e27b5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.062489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.063191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.063516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.063813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.074644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:05.079699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:05.080611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:05.081411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 160/327] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:05.088864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.146217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b5104-4b80-455c-b445-083d316ece62 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.170058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:05.196087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f99d-9164-42a1-8a9c-43c2bb218421 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.201407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.202693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.203189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.203709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.228729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:05.232322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.233057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.233433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.233801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.234623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:05.235030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:05.235150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:05.236211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:05.237223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:05.239094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:05.248180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 168/328] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:05.249075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 161/329] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:05.249549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.255854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.316209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:05.362775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec4cc4-a875-4939-9945-30f966736c62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.399654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.400499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.400891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.401260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] PATH_INFO: `/v3/users/47927500ff8b4391be14cac3d3e1538e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.402546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=47927500ff8b4391be14cac3d3e1538e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:05.404123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:05.406141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 162/330] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:05 2022] GET /identity/v3/users/47927500ff8b4391be14cac3d3e1538e => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:05.411924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.426557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:05.494056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc50638-8aba-434a-8d29-e9f6a38caaf7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.536609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.537240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.537522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.537800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.538803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:05.539103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:05.540932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:05.553679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 169/331] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:05.564230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.608030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33964c-9df6-4084-b1a7-c217c437a3f0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.696493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:05.700425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.701260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.701520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.701892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.702803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] RBAC: Adding query filter params (role.id=3f8debab12324e0da238b80499b5eafe, user.id=47927500ff8b4391be14cac3d3e1538e, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:05.703132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:05.704698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:05.710199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f9a79-e003-45cc-bb94-1ae7dbf06d17 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.735800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 163/332] 10.222.0.30 () {56 vars in 1308 bytes} [Fri Aug 5 05:00:05 2022] GET /identity/v3/role_assignments?user.id=47927500ff8b4391be14cac3d3e1538e&scope.system=all&role.id=3f8debab12324e0da238b80499b5eafe => generated 510 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:05.766391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c674311-d9c4-4ac2-85b6-843dbfe9c48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.767789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c674311-d9c4-4ac2-85b6-843dbfe9c48c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.768249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c674311-d9c4-4ac2-85b6-843dbfe9c48c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.768668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c674311-d9c4-4ac2-85b6-843dbfe9c48c None admin] PATH_INFO: `/v3/users/02022474ce36470a9b56d53fb5baa295` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.770683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c674311-d9c4-4ac2-85b6-843dbfe9c48c None admin] RBAC: Authorizing `identity:get_user(user_id=02022474ce36470a9b56d53fb5baa295)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:05.772894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c674311-d9c4-4ac2-85b6-843dbfe9c48c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:05.775293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 170/333] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:05 2022] GET /identity/v3/users/02022474ce36470a9b56d53fb5baa295 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:05.781413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:05.854385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:05.899502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516889-2e57-4187-b274-bd85f08ef1ca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:05.938552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47212df-c6f7-4250-9ed2-07a69a37ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:05.939490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47212df-c6f7-4250-9ed2-07a69a37ff05 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:05.939981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47212df-c6f7-4250-9ed2-07a69a37ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:05.940407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47212df-c6f7-4250-9ed2-07a69a37ff05 None admin] PATH_INFO: `/v3/system/users/02022474ce36470a9b56d53fb5baa295/roles/f1bde6b54454477cb807392e68475a8c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:05.942079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47212df-c6f7-4250-9ed2-07a69a37ff05 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=02022474ce36470a9b56d53fb5baa295, role_id=f1bde6b54454477cb807392e68475a8c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:05.943690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47212df-c6f7-4250-9ed2-07a69a37ff05 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:05.965904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 164/334] 10.222.0.30 () {58 vars in 1334 bytes} [Fri Aug 5 05:00:05 2022] PUT /identity/v3/system/users/02022474ce36470a9b56d53fb5baa295/roles/f1bde6b54454477cb807392e68475a8c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:07.209108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777cf7-7277-4c06-a06e-2db6380b9aa4 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.210111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777cf7-7277-4c06-a06e-2db6380b9aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.210582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777cf7-7277-4c06-a06e-2db6380b9aa4 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.212214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 171/335] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:07.219807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.219807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.219807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.240802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:07.242312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:07.320138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 165/336] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:07.361239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777cf7-7277-4c06-a06e-2db6380b9aa4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:07.493469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777cf7-7277-4c06-a06e-2db6380b9aa4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:07.539250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:07.539941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.540223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.540496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.557477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:07.562972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:07.563871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:07.564642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 172/337] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:07.570357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:07.639578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:07.688508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5bdfd-c8fe-4c80-93d4-cd21d9a6c904 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:07.726758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:07.727803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.728274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.728733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.729651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:07.730139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:07.731731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3e748-7825-487b-ab68-a6b92134b7e0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:07.741298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 166/338] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:07.747884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:07.763338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:07.857795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f3add-a86f-4043-8f73-89c0da62dd2c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:07.891650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:07.892319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.892600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.892871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.914918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563ac6-00bc-4298-9206-7f903ec68b9a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.915233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563ac6-00bc-4298-9206-7f903ec68b9a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.915471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563ac6-00bc-4298-9206-7f903ec68b9a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.916853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 167/339] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:07.928736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14494bc9-d6b0-4bc4-a08a-bd152f918e32 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:07.929031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14494bc9-d6b0-4bc4-a08a-bd152f918e32 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:07.929286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14494bc9-d6b0-4bc4-a08a-bd152f918e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:07.964038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-14494bc9-d6b0-4bc4-a08a-bd152f918e32 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:07.966096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14494bc9-d6b0-4bc4-a08a-bd152f918e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:07.988465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:07.990434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:08.006456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.102689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 173/340] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 05:00:08.114089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.177136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 168/341] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:08.203484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0d2c1-eb6b-4c66-8689-79164edcd867 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.203807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0d2c1-eb6b-4c66-8689-79164edcd867 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.204074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0d2c1-eb6b-4c66-8689-79164edcd867 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.205463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 169/342] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:08.212606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.212888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.213163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.221894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdce8f7-b2eb-4dfc-90fb-489111520954 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:08.252214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.257339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:08.258824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:08.264700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:08.267300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.271244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.271751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.274448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:08.276123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:08.277999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 174/343] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:08.281355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.372209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.416700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 170/344] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:08.417975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.459775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d14ee9-2f48-41d7-a8ca-c9bf9c15e6a3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:08.499878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7ef7f-ca5d-4f5b-9bfb-e1f9e09ff2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:08.500480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7ef7f-ca5d-4f5b-9bfb-e1f9e09ff2aa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.500763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7ef7f-ca5d-4f5b-9bfb-e1f9e09ff2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.501039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7ef7f-ca5d-4f5b-9bfb-e1f9e09ff2aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.511520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7ef7f-ca5d-4f5b-9bfb-e1f9e09ff2aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a8c7ef7f-ca5d-4f5b-9bfb-e1f9e09ff2aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:08.517458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:08.518318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:08.519085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 175/345] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:08.519934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bb8ba-aeea-40fe-990b-b9b6aa474b35 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.520483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bb8ba-aeea-40fe-990b-b9b6aa474b35 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.520811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bb8ba-aeea-40fe-990b-b9b6aa474b35 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.522052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 176/346] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:08.522790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bb8ba-aeea-40fe-990b-b9b6aa474b35 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.541076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522939c-9b91-41e1-b896-04a687d16334 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:08.580225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.585996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:08.587402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.587998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.588402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.595979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:08.597790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:08.601395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 171/347] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:08.602638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.649447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bb8ba-aeea-40fe-990b-b9b6aa474b35 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:08.663845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.695586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47444cd6-dd43-45bd-9e39-34a176a10a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:08.696328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47444cd6-dd43-45bd-9e39-34a176a10a74 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.696738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47444cd6-dd43-45bd-9e39-34a176a10a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.697024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47444cd6-dd43-45bd-9e39-34a176a10a74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.716266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47444cd6-dd43-45bd-9e39-34a176a10a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-47444cd6-dd43-45bd-9e39-34a176a10a74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:08.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:08.724271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:08.725146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 177/348] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:08.731893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.732220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.732500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.753659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14e08b-f5d0-4561-9219-2203f071ba37 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:08.755214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:08.756646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:08.801969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:08.803116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:08.803457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:08.803811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:08.804604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:08.804971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:08.806427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:08.818067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 172/349] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:08.819691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.838365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 178/350] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:08.839553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:08.892915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.958432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766440e8-ce8a-4435-b0e1-5f807b8273f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:08.968646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:08.970599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a306-c8c5-4729-b4c3-2b8174367ad3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.009906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.010422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.010915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.011394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.014351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=c9ad666675744c88976e6b45e1415a3a) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:09.014920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:09.016635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:09.020553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.021757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.022039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.022321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.023320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:09.023625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:09.025087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:09.025536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 173/351] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=3f8debab12324e0da238b80499b5eafe => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:09.026560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.036240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 179/352] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:09.037182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.159808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:09.175956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b13cf4-fff7-4d5b-a074-94ccfb63d447 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.223420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.224047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.224312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.224589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.224785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be70f8-9ed1-4439-80be-de8e65bc1eba None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.236341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:09.242005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:09.242891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:09.243659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 180/353] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:09.244335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.264616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf45f0b-78d8-4e52-a06b-3c4d04d33aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.265250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45f0b-78d8-4e52-a06b-3c4d04d33aaf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.265518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45f0b-78d8-4e52-a06b-3c4d04d33aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.265784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45f0b-78d8-4e52-a06b-3c4d04d33aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.315462 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:09.349574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf45f0b-78d8-4e52-a06b-3c4d04d33aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:09.351055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf45f0b-78d8-4e52-a06b-3c4d04d33aaf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:09.377644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb27f-4244-4cd9-9e30-dc92a3d64ba9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.420123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.420929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.421222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.421608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.446526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:09.480312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 174/354] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:09.481562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d60c-b078-46bf-9cee-35d0c7a660c5 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.481851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d60c-b078-46bf-9cee-35d0c7a660c5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.482168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d60c-b078-46bf-9cee-35d0c7a660c5 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.483732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 175/355] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:09.485370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7a6ed-c473-46e6-9b6e-05ae9e8c4435 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.485775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7a6ed-c473-46e6-9b6e-05ae9e8c4435 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.486051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7a6ed-c473-46e6-9b6e-05ae9e8c4435 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.487536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 176/356] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:09.488708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7a6ed-c473-46e6-9b6e-05ae9e8c4435 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.518191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:09.519655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:09.582741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:09.624161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7a6ed-c473-46e6-9b6e-05ae9e8c4435 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.655444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 181/357] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:09.656753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.668503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.669501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.669955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.670360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.671083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:09.671546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:09.673423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f622564-1fa4-4252-868d-7b804cf19c0b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:09.684241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 177/358] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:09.686090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86f0c-49f9-4861-bbb4-a6e35fea1193 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.686388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86f0c-49f9-4861-bbb4-a6e35fea1193 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.686667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86f0c-49f9-4861-bbb4-a6e35fea1193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.708129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-77f86f0c-49f9-4861-bbb4-a6e35fea1193 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:09.710204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77f86f0c-49f9-4861-bbb4-a6e35fea1193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:09.724622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:09.789451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183efa53-f9ec-4bf4-aaa0-70fdda706ae2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.795093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 178/359] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:09.796401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.796691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.796973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.821704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:09.823014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:09.831475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:09.832654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:09.832943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:09.833225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:09.835412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:09.837248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:09.839435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 182/360] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:09.840756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.888924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:09.898930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 179/361] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:09.900194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:09.967370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6ef9d-541c-4a2d-b268-f01e8c494583 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:09.998100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.017354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.018017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.018298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.018574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.020372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.021394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.022664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9acc28-f2fa-4b0b-b12e-19a23abaa8ee None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.023245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 183/362] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.023919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.068164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.069152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.069639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.070093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.141793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec6ab5-20de-416e-83e4-b9f1bd4f9709 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.152506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.165027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.166553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.190166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.191583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.191915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.192194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.207935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:10.219829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:10.220784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 184/363] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:10.221709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.285355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.288178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 180/364] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:10.289760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.347334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eca9a-d682-452e-be83-1e3f3a48a6c9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.399415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.400172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.400450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.400898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.402293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.402438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.403899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.405672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 185/365] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.406349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.414241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839db1-aef5-4ed8-b5c7-c9a2b2ac5301 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.455492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.456442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.456862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.457269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.469603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:10.474562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:10.475443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:10.476368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 181/366] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:10.477503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.526238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceeac3-1463-4038-aa68-5747ba1680fc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.551504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.583182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.583922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.584205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.584510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.585631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.587073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.589761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 186/367] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.589761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.620006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8cd2e-93fb-44c5-99c4-674ef65cb472 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.666670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.667400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.667747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.668122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.668917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:10.669284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.670762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.681169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 182/368] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.682011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.698725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.718657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028201f-7426-497e-91eb-332f8f6e04ed None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.759690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.760431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.760717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.761003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.763881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.765359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.767345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 187/369] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.768024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.805249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667897f1-7c5c-4f24-86ca-1bddf54f6a87 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.843493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:10.859482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.860557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.860990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.861306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.862060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=c9ad666675744c88976e6b45e1415a3a) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:10.862732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.863923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.874412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 183/370] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.876363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.903330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec407c-54e6-4217-8e93-fbd3359fc4e9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:10.946480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:10.947298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:10.947644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:10.948072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:10.948880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:10.949259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:10.950803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:10.962922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 188/371] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:10.963618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:10.995660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.035318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7835bdb-0117-4695-9da3-d20ed1e088c7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.077000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.077649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.077915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.078187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.078984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:11.079296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.081585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.087913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 184/372] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:11.088758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.102519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a4d4-7a9c-45ce-bea7-1614c1415f49 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.118965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.145940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.146519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.146804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.147080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] PATH_INFO: `/v3/users/02022474ce36470a9b56d53fb5baa295` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.148400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] RBAC: Authorizing `identity:get_user(user_id=02022474ce36470a9b56d53fb5baa295)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.149905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.152169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 189/373] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:00:10 2022] GET /identity/v3/users/02022474ce36470a9b56d53fb5baa295 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:11.153064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.199566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132ae46-970f-4494-8311-ba3e644798da None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.239502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.240210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.240492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.240771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.242012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.243538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.245625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 185/374] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:11.246479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.264988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.276413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacdde2-9126-479d-bb78-7862235ac1e1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.316650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba4f09-e6c0-4aea-8110-5494be8837ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.317414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba4f09-e6c0-4aea-8110-5494be8837ba None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.317567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba4f09-e6c0-4aea-8110-5494be8837ba None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.318035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba4f09-e6c0-4aea-8110-5494be8837ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.370078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5137c35-9f85-466e-ab95-61fd3c2b1bb1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.377723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.398757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba4f09-e6c0-4aea-8110-5494be8837ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.400348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba4f09-e6c0-4aea-8110-5494be8837ba None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.421502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.421787 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.422061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.422863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] RBAC: Adding query filter params (role.id=f1bde6b54454477cb807392e68475a8c, user.id=02022474ce36470a9b56d53fb5baa295, scope.system=all) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:11.423175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.424679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e75b8-b88f-4192-bd9d-2d247f0f7a6f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.441760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 186/375] 10.222.0.30 () {56 vars in 1308 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/role_assignments?user.id=02022474ce36470a9b56d53fb5baa295&scope.system=all&role.id=f1bde6b54454477cb807392e68475a8c => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:11.443223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffcdfb-88f7-49f7-8d4e-bb3d9ea4735d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.443518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffcdfb-88f7-49f7-8d4e-bb3d9ea4735d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.443841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffcdfb-88f7-49f7-8d4e-bb3d9ea4735d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.450421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 187/376] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:11.450421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffcdfb-88f7-49f7-8d4e-bb3d9ea4735d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.530758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 190/377] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:11.532644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.533136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.533606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.539578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.555676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:11.557418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:11.589539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffcdfb-88f7-49f7-8d4e-bb3d9ea4735d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.647004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.648265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.648828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.649295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.650494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:11.651002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.652716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.659828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 188/378] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=3f8debab12324e0da238b80499b5eafe => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:11.665885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.731810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 191/379] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:11.788349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.822026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.831670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae651e18-dee4-4a8a-ba2d-0cfa6573187a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:11.870176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:11.879003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:11.880022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:11.880761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:11.881747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] PATH_INFO: `/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:11.885256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=4ea26c15401c49d28fd797d4ee85aa6c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:11.887365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:11.894752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 189/380] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:11.896569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:11.977254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d701c8-9c88-4dd3-9920-fbb9b6e63705 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.046062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.046723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.047002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.047271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.064041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:12.068950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:12.069808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:12.070977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 192/381] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:12.076249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:12.078811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dcec9-75d4-4529-92ff-d164a0bcd2d0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.140515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.141656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.142206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.142677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] PATH_INFO: `/v3/groups/f7a60598a30041ea9616052223d1ba2d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.151231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] RBAC: Authorizing `identity:get_group(group_id=f7a60598a30041ea9616052223d1ba2d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:12.152900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:12.154478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:12.161715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 190/382] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:11 2022] GET /identity/v3/groups/f7a60598a30041ea9616052223d1ba2d => generated 229 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:12.163611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:12.183591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890144b-ea11-4937-bd1a-5f66fde06063 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.219948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.220578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.220857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.221134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.221858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:12.222158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:12.223643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:12.232601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 193/383] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:12.238163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:12.262934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:12.323942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d5c7-c3a0-4e59-b011-b03195250608 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.341992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3281ac8-3312-4c84-b1be-b5f9e80f93dc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.378728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.379522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.379930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.380316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.381552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:12.382352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.383178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:12.383622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.384053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.384455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.385374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] RBAC: Adding query filter params (group.id=f7a60598a30041ea9616052223d1ba2d, role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=4ea26c15401c49d28fd797d4ee85aa6c) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:12.385557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 194/384] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:12 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:12.385973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:12.391207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:12.391580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:12.394134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:12.399404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 191/385] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:12 2022] GET /identity/v3/role_assignments?group.id=f7a60598a30041ea9616052223d1ba2d&scope.project.id=4ea26c15401c49d28fd797d4ee85aa6c&role.id=47aca6aac2934a5cb816872abfc94994 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:12.505595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c613-1ef5-409e-879f-d1f85c2ca078 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.543228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c32836-36c7-459a-b280-9dc3f6cae73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.543935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c32836-36c7-459a-b280-9dc3f6cae73b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.544221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c32836-36c7-459a-b280-9dc3f6cae73b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.544835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c32836-36c7-459a-b280-9dc3f6cae73b None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.546091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c32836-36c7-459a-b280-9dc3f6cae73b None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:12.547540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c32836-36c7-459a-b280-9dc3f6cae73b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:12.549642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 195/386] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:12 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:12.556047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:12.683739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5e6b-8e29-417d-96f0-35115ac0a069 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:12.714175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:12.723957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeceff5-8894-4e3f-aa9d-abad68f4e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:12.724705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeceff5-8894-4e3f-aa9d-abad68f4e81a None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:12.724992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeceff5-8894-4e3f-aa9d-abad68f4e81a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:12.725333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeceff5-8894-4e3f-aa9d-abad68f4e81a None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a/groups/eb171bc025c64cc096a961b2c560ec61/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:12.727551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeceff5-8894-4e3f-aa9d-abad68f4e81a None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ad666675744c88976e6b45e1415a3a, group_id=eb171bc025c64cc096a961b2c560ec61, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:12.729368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeceff5-8894-4e3f-aa9d-abad68f4e81a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:12.747981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 192/387] 10.222.0.30 () {58 vars in 1442 bytes} [Fri Aug 5 05:00:12 2022] PUT /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a/groups/eb171bc025c64cc096a961b2c560ec61/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:13.690658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321deb8-a15b-40a3-bd68-08b5623dc763 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:13.691227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321deb8-a15b-40a3-bd68-08b5623dc763 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:13.691227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321deb8-a15b-40a3-bd68-08b5623dc763 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:13.692872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 196/388] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:13.697773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:13.698167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:13.698343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:13.724771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:13.725971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:13.772865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f05cb-e065-42d9-9c6f-08059a061acf None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:13.773201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f05cb-e065-42d9-9c6f-08059a061acf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:13.773422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f05cb-e065-42d9-9c6f-08059a061acf None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:13.774787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 197/389] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:13.786792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3d48c-3e3f-4434-9ee4-107a924c6d7b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:13.787082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3d48c-3e3f-4434-9ee4-107a924c6d7b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:13.787363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3d48c-3e3f-4434-9ee4-107a924c6d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:13.818792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fca3d48c-3e3f-4434-9ee4-107a924c6d7b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:13.820019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fca3d48c-3e3f-4434-9ee4-107a924c6d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:13.823712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 193/390] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:13.866207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:13.916407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:13.925327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 198/391] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:13.965016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ac88a-8996-43a4-bd68-6f7cbe194cda None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:13.965295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ac88a-8996-43a4-bd68-6f7cbe194cda None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:13.965480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ac88a-8996-43a4-bd68-6f7cbe194cda None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:13.966906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 199/392] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:00:13.971894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:13.972177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:13.972451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:13.996109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:13.999785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:14.019788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14badd42-26d6-4cb6-9fc8-e1f749ba5a5c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.035890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.055726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.056471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.056814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.057258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.068433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:14.073873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:14.075057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 194/393] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:14.076339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.084393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 200/394] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:14.085796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.207900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f67b6-d03a-4ce8-a49b-086c32577768 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.219429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fac12b-e6de-4d90-92d1-612e7f4d3dbc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.244871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.252156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.252793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.253073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.253440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.254264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:14.254571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.255967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.265681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.266353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.266620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.266895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.269872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 201/395] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.270899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.281095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:14.282545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:14.283395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:14.284228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 195/396] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:14.284879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.383564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71f20c-623f-4712-996f-a6edea9cc0d7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.384235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.400675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898f2f-9e7f-4561-a9ba-fb963ff97547 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.420199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.421287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.421819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.422386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.433719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:14.440155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:14.441031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:14.441831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:14.442466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.442466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.442466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.442466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.443974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.445141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 202/397] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:14.445488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.446335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.456268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 196/398] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.456268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.511509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.559192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dd49d-1fdc-4e62-b617-16ea3d493ea0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.565166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3032ea8-5604-44f1-80f4-7faddf98b5e7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.566581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.594064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.594700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.594980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.595251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.595995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:14.596301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.597677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.604376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.605289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.605774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.606237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 197/399] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.606715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.606823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.608243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:14.608763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.610498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.619893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 203/400] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.621011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.713064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5ff1e-7d5a-4da8-b473-4060de372a95 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.730242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb6603-b0c5-450c-8227-3ce1b8572276 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.746746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.749367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.750056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.750500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.750775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.752008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.753503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.759985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 198/401] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.759985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.778924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.779942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.780415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.780946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.782377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.784030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.786180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 204/402] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.787185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.803398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:14.870658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12081c1a-e6b4-4ffd-ac4b-0a9da81141ce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.894093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555eba8-49ad-4673-9889-753b36148f7a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:14.904991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.905725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.906009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.906273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.907519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.908993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.915488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 199/403] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:14.916452 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:14.931974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c97b8-1787-492f-b722-1b7e72df3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:14.932715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c97b8-1787-492f-b722-1b7e72df3829 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.932996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c97b8-1787-492f-b722-1b7e72df3829 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.933271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c97b8-1787-492f-b722-1b7e72df3829 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a/groups/eb171bc025c64cc096a961b2c560ec61/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.935589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c97b8-1787-492f-b722-1b7e72df3829 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9ad666675744c88976e6b45e1415a3a, group_id=eb171bc025c64cc096a961b2c560ec61, role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:14.937085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c97b8-1787-492f-b722-1b7e72df3829 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:14.952183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 205/404] 10.222.0.30 () {58 vars in 1442 bytes} [Fri Aug 5 05:00:14 2022] PUT /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a/groups/eb171bc025c64cc096a961b2c560ec61/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:14.962215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5472d36-37c8-476a-8cfb-122ebea50d0d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:14.963072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5472d36-37c8-476a-8cfb-122ebea50d0d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:14.963355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5472d36-37c8-476a-8cfb-122ebea50d0d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:14.965971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 206/405] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:14.966958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5472d36-37c8-476a-8cfb-122ebea50d0d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.032453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:15.052033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec215-5fb7-488f-bac2-2407287c2c02 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.056294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:15.102629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df4e68-3064-4a10-9085-2f2633d8e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.103968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df4e68-3064-4a10-9085-2f2633d8e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.104734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df4e68-3064-4a10-9085-2f2633d8e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.105676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df4e68-3064-4a10-9085-2f2633d8e0c2 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.107752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df4e68-3064-4a10-9085-2f2633d8e0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:15.110204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df4e68-3064-4a10-9085-2f2633d8e0c2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:15.112953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 200/406] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:15.114688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.115225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.115704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.116489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5472d36-37c8-476a-8cfb-122ebea50d0d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.136812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:15.138011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:15.157071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.157071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.157071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.157408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.159297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:15.160815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:15.162511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 207/407] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:14 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:15.163244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.214239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 201/408] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:15.215627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.274245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882f686-15a0-4930-940a-846e65b96728 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.284559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:15.308679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2a9c0-2579-47ef-afb3-2f3de1a476ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.309363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a9c0-2579-47ef-afb3-2f3de1a476ae None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.309650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a9c0-2579-47ef-afb3-2f3de1a476ae None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.309937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a9c0-2579-47ef-afb3-2f3de1a476ae None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/groups/eb171bc025c64cc096a961b2c560ec61/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.312070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2a9c0-2579-47ef-afb3-2f3de1a476ae None admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c4224b99e4c15856ab6c5e76cd861, group_id=eb171bc025c64cc096a961b2c560ec61, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:15.314161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2a9c0-2579-47ef-afb3-2f3de1a476ae None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:15.334173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49885a0-1139-44ab-8c30-fcd9eaaac628 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.335885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 208/409] 10.222.0.30 () {58 vars in 1442 bytes} [Fri Aug 5 05:00:15 2022] PUT /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/groups/eb171bc025c64cc096a961b2c560ec61/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:15.336314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:15.337545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61721266-ccf9-4d8f-9c51-10c3ee737c14 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.337820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61721266-ccf9-4d8f-9c51-10c3ee737c14 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.338168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61721266-ccf9-4d8f-9c51-10c3ee737c14 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.340594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 209/410] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:15.344886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61721266-ccf9-4d8f-9c51-10c3ee737c14 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.381850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7de89-7e41-4c84-9007-43b4b55686da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.382584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7de89-7e41-4c84-9007-43b4b55686da None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.382903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7de89-7e41-4c84-9007-43b4b55686da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.383191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7de89-7e41-4c84-9007-43b4b55686da None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/groups/eb171bc025c64cc096a961b2c560ec61/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.385993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7de89-7e41-4c84-9007-43b4b55686da None admin] RBAC: Authorizing `identity:create_grant(project_id=8f5c4224b99e4c15856ab6c5e76cd861, group_id=eb171bc025c64cc096a961b2c560ec61, role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:15.387645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7de89-7e41-4c84-9007-43b4b55686da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:15.407672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 202/411] 10.222.0.30 () {58 vars in 1442 bytes} [Fri Aug 5 05:00:15 2022] PUT /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861/groups/eb171bc025c64cc096a961b2c560ec61/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:15.409168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.410029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.410327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.438095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:15.438095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:15.534364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 203/412] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:15.546580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61721266-ccf9-4d8f-9c51-10c3ee737c14 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.561830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.590316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.591150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.591617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.592007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.609990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:15.616863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:15.617964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:15.618960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 210/413] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:15.625699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.660559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:15.675116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:15.686465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a056b-8de1-4a8e-9070-cd032564e684 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.747290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.748121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.748335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.748610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.768023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:15.771150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca1b6c-f92b-4ac8-bac2-9e2394b64d41 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:15.781489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:15.782592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:15.783604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 204/414] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:15.789872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.827785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:15.830736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:15.831025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:15.831301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:15.832338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:15.832654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:15.834065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:15.851291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 211/415] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:15.856694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:15.972662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc1385-8b3a-4a22-9a1f-38bc1a2414e5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:16.028168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:16.038524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:16.039201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:16.039441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:16.039720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:16.040473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:16.040783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:16.042183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:16.045657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654ea3-a2fa-4b3f-bf73-ba3509457762 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:16.055048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 205/416] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:16.062215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:16.091555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:16.091963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:16.092812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:16.093092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:16.093371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] PATH_INFO: `/v3/groups/f7a60598a30041ea9616052223d1ba2d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:16.096774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] RBAC: Authorizing `identity:get_group(group_id=f7a60598a30041ea9616052223d1ba2d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:16.098231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:16.099887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 212/417] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:15 2022] GET /identity/v3/groups/f7a60598a30041ea9616052223d1ba2d => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:16.105432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:16.265504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ec2e2-5d81-44d0-832c-7288c2fca90c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:16.285754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06967154-c03a-4d27-a83b-3197324c0ec1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:16.305195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:16.305847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:16.306123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:16.306393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] PATH_INFO: `/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:16.307675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] RBAC: Authorizing `identity:get_project(project_id=4ea26c15401c49d28fd797d4ee85aa6c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:16.310928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:16.313760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 213/418] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:16 2022] GET /identity/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:16.320732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:16.352333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:16.376476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:16.377130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:16.377416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:16.377703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:16.451343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eed6d-c086-4068-8046-b006e3ec5de0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:16.532696 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:16.533190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:16.534398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:16.536987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:16.537705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:16.538373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] PATH_INFO: `/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c/groups/f7a60598a30041ea9616052223d1ba2d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:16.542657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ea26c15401c49d28fd797d4ee85aa6c, group_id=f7a60598a30041ea9616052223d1ba2d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:16.544961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:16.569661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:16.577658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 214/419] 10.222.0.30 () {58 vars in 1442 bytes} [Fri Aug 5 05:00:16 2022] PUT /identity/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c/groups/f7a60598a30041ea9616052223d1ba2d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:16.711222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 206/420] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:16.717399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:16.840063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:16.858571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ad1f-9ceb-4bd8-a343-43d309414481 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:16.903345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29081054-d419-435f-b0cd-b59592542015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:16.904513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081054-d419-435f-b0cd-b59592542015 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:16.905019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081054-d419-435f-b0cd-b59592542015 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:16.905489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081054-d419-435f-b0cd-b59592542015 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:16.906865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29081054-d419-435f-b0cd-b59592542015 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:16.908546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29081054-d419-435f-b0cd-b59592542015 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:16.910772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 215/421] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:16 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:16.916863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:17.029412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960a7b9-57ab-4ae2-80ac-88d9c5a8783a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:17.043950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:17.072962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f08dd-5e17-49cd-a0f9-b624c2389aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:17.073627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f08dd-5e17-49cd-a0f9-b624c2389aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.073912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f08dd-5e17-49cd-a0f9-b624c2389aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.074185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f08dd-5e17-49cd-a0f9-b624c2389aa9 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.075355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f08dd-5e17-49cd-a0f9-b624c2389aa9 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:17.076874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f08dd-5e17-49cd-a0f9-b624c2389aa9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:17.078769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 207/422] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:16 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:17.084856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29081054-d419-435f-b0cd-b59592542015 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:17.199033 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29081054-d419-435f-b0cd-b59592542015 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:17.242844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:17.243475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.243754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.244074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.244879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=c9ad666675744c88976e6b45e1415a3a) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:17.245190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:17.246639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f50c9-bf8c-4000-b315-579cc8b12985 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:17.253468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 216/423] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:17 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=3f8debab12324e0da238b80499b5eafe => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:17.325246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2992f3-937c-4837-8dba-091134ca07f8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.325649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2992f3-937c-4837-8dba-091134ca07f8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.325872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2992f3-937c-4837-8dba-091134ca07f8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.327485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 208/424] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:17.332767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be7ca4-7f31-4833-b80d-3f97d3364a86 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.333060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be7ca4-7f31-4833-b80d-3f97d3364a86 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.333343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be7ca4-7f31-4833-b80d-3f97d3364a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.352683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d0be7ca4-7f31-4833-b80d-3f97d3364a86 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:17.353893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d0be7ca4-7f31-4833-b80d-3f97d3364a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:17.369581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:17.427949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 217/425] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:17.451103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2992f3-937c-4837-8dba-091134ca07f8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:17.597955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2992f3-937c-4837-8dba-091134ca07f8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:17.599477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8aff6-21e9-438e-8e71-cd236035b852 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.599791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8aff6-21e9-438e-8e71-cd236035b852 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.600080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8aff6-21e9-438e-8e71-cd236035b852 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.605139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 218/426] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:17.609448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.609737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.610021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.631689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:17.633320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:17.646943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:17.648031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.648506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.648991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.662736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:17.668167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:17.669025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:17.669837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:17.670971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 209/427] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:17.675889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:17.734732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 219/428] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:17.739464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:17.765521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:17.819863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d588-5319-49c6-ad88-7981300ac70b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:17.858593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:17.861868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:17.862752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.863272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.863752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.865025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:17.865577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:17.867712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c611bc-dfc0-4910-89ee-dfaad52df35c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:17.878909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 210/429] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:17.880866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a96b9-0f53-4a84-97d1-383fde0e6201 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.881350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a96b9-0f53-4a84-97d1-383fde0e6201 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.881815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a96b9-0f53-4a84-97d1-383fde0e6201 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.884567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ef65a-a593-4f60-b6fe-d99d0e9e9476 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:17.884829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 211/430] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:17.885798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a96b9-0f53-4a84-97d1-383fde0e6201 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:17.923153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84921b-f558-4271-b144-68bc9bdc6de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:17.924490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84921b-f558-4271-b144-68bc9bdc6de0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.924775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84921b-f558-4271-b144-68bc9bdc6de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.925052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84921b-f558-4271-b144-68bc9bdc6de0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.937396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84921b-f558-4271-b144-68bc9bdc6de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9d84921b-f558-4271-b144-68bc9bdc6de0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:17.942363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:17.943270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 220/431] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:17.948591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:17.949308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:17.949308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:17.968964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:17.970745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:18.016647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a96b9-0f53-4a84-97d1-383fde0e6201 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.021225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.062455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 221/432] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:18.063404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.065697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.067425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.067946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.068423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.139389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.142024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.143645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.170121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6e3c-983a-465b-bca7-24083ee84a21 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.205276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.206016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.206304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.206586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.207322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:18.207804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.209199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.218515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 222/433] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:18.223704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.249621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 212/434] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:18.250301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.254314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.339113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d3d9-6bfb-43be-b96b-bf824711b9d2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.358079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577e2f-4248-48f8-a89d-19fe446e15b0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.369568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.376126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.376751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.377043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.377298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.387880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:18.392511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:18.392744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:18.393585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:18.394999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:18.395676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.395676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.395676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.395878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 223/435] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:18.395878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.464305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.465879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.469432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.505964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f73bdb-657e-4ea7-95c4-2bb6b29a5703 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.542530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.543173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.543452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.543723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] PATH_INFO: `/v3/projects/c9ad666675744c88976e6b45e1415a3a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.545037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] RBAC: Authorizing `identity:get_project(project_id=c9ad666675744c88976e6b45e1415a3a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.546497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.548403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 224/436] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/projects/c9ad666675744c88976e6b45e1415a3a => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:18.553296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.575578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 213/437] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:18.575578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.617771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.658029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1504a0b-e667-4719-aee5-356d09cade29 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.692049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.692677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.692958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.693297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.694016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:18.694322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.695703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.698405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b067-f1a5-4f15-9950-31ca9c8642c2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.702053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.708825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 225/438] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:18.708825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.741358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.742035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.742318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.742593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.743861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.745479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.747229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 214/439] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:18.748021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.815780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680e411-9b71-45a3-a446-fe8b762379d7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.837868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.849771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.850429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.850715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.850978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.852165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.853605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:18.854463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f36b55-8da6-4878-8d6a-b35dc1f2ddfc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.855245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 226/440] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:18.856018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:18.893499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b7712-c63e-470d-9edd-6106559192cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:18.894121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b7712-c63e-470d-9edd-6106559192cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:18.894400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b7712-c63e-470d-9edd-6106559192cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:18.894677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b7712-c63e-470d-9edd-6106559192cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:18.942510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:18.966519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75947b-df58-44c6-ae29-9687027e4f5d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:18.974092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b7712-c63e-470d-9edd-6106559192cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:18.975863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b7712-c63e-470d-9edd-6106559192cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.003345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.004103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.004447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.004952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.005583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=c9ad666675744c88976e6b45e1415a3a) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:19.005903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.007340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.016265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 227/441] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=c9ad666675744c88976e6b45e1415a3a&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:19.025617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.095816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 215/442] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:19.097214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f1c43-d2ae-4048-b042-72c52aec3372 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.097600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f1c43-d2ae-4048-b042-72c52aec3372 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.097957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f1c43-d2ae-4048-b042-72c52aec3372 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.099563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 216/443] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:19.101648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f1c43-d2ae-4048-b042-72c52aec3372 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.110313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:19.160544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b578d-c903-42d9-9c0a-31563a974ee5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.181526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:19.203065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db280c1f-d9a5-4700-b6b4-842deb2dd67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.203648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db280c1f-d9a5-4700-b6b4-842deb2dd67d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.203960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db280c1f-d9a5-4700-b6b4-842deb2dd67d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.204236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db280c1f-d9a5-4700-b6b4-842deb2dd67d None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.205479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db280c1f-d9a5-4700-b6b4-842deb2dd67d None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.207250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db280c1f-d9a5-4700-b6b4-842deb2dd67d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.210652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 228/444] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:19.210652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.211038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.211109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.218778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f1c43-d2ae-4048-b042-72c52aec3372 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.229803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:19.230999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:19.257378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.258040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.258316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.258756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] PATH_INFO: `/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.259836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] RBAC: Authorizing `identity:get_project(project_id=8f5c4224b99e4c15856ab6c5e76cd861)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.261252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.263343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 217/445] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/projects/8f5c4224b99e4c15856ab6c5e76cd861 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:19.264145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.305926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 229/446] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:19.306805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.373789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:19.375931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63430b3f-7541-4d51-9497-a703aee742b4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.415099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.415735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.416097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.416376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.417168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=3f8debab12324e0da238b80499b5eafe, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:19.417483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.418953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.424733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 218/447] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=3f8debab12324e0da238b80499b5eafe => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:19.425394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90081c-5584-41a3-b850-0300d5c72c34 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.425501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.429297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:19.467280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.467903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.468192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.468466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] PATH_INFO: `/v3/groups/eb171bc025c64cc096a961b2c560ec61` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.469676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] RBAC: Authorizing `identity:get_group(group_id=eb171bc025c64cc096a961b2c560ec61)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.471688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.474009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 230/448] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/groups/eb171bc025c64cc096a961b2c560ec61 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:19.480851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.547936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd464c34-56f5-4253-96f6-fb818b709e13 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.588408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.589066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.589361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.589636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.593651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83130a29-fff3-4a3b-a01f-a9b69891b884 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.602339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:19.608626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:19.609484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:19.610264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 219/449] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:19.615741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:19.616973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.636428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.637065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.637342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.637612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.638537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] RBAC: Adding query filter params (group.id=eb171bc025c64cc096a961b2c560ec61, role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:19.638852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.640329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.646558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 231/450] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/role_assignments?group.id=eb171bc025c64cc096a961b2c560ec61&scope.project.id=8f5c4224b99e4c15856ab6c5e76cd861&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:19.695585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:19.748757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e127c8-f305-4c95-b5a2-fd304eb8723f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.786549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.787204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.787516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.787786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.788608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:19.788920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:19.790362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:19.801010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 220/451] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:19.806995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:19.913274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c658d-43e5-4259-8ddc-344faf55e995 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:19.946663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:19.947288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:19.947566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:19.947875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:19.998933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:20.014843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:20.016352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:20.113972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 232/452] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:20.120019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:20.223168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7163b-d879-447b-ab31-baf88a0891cb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:20.248375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:20.257591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:20.258157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:20.258440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:20.258713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] PATH_INFO: `/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:20.259960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=4ea26c15401c49d28fd797d4ee85aa6c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:20.261403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:20.263026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 221/453] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:00:20 2022] GET /identity/v3/projects/4ea26c15401c49d28fd797d4ee85aa6c => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:20.271704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:20.361463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:20.380013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b74d7-4660-4687-8879-4f8c28fbf428 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:20.415379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fee77-d140-46ed-b7a7-faa532df07ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:20.416045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fee77-d140-46ed-b7a7-faa532df07ff None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:20.416330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fee77-d140-46ed-b7a7-faa532df07ff None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:20.416598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fee77-d140-46ed-b7a7-faa532df07ff None admin] PATH_INFO: `/v3/groups/f7a60598a30041ea9616052223d1ba2d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:20.417791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fee77-d140-46ed-b7a7-faa532df07ff None admin] RBAC: Authorizing `identity:get_group(group_id=f7a60598a30041ea9616052223d1ba2d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:20.419232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fee77-d140-46ed-b7a7-faa532df07ff None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:20.420926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 233/454] 10.222.0.30 () {56 vars in 1180 bytes} [Fri Aug 5 05:00:20 2022] GET /identity/v3/groups/f7a60598a30041ea9616052223d1ba2d => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:20.427728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:20.543423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778a0e6-c450-47c0-9fa4-5cfd1f902bd2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:20.585075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:20.585682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:20.585962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:20.586233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:20.587010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] RBAC: Adding query filter params (group.id=f7a60598a30041ea9616052223d1ba2d, role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=4ea26c15401c49d28fd797d4ee85aa6c) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:20.587320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:20.588776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed40f5-4781-4014-afe2-f466cdc8c442 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:20.594664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 222/455] 10.222.0.30 () {56 vars in 1376 bytes} [Fri Aug 5 05:00:20 2022] GET /identity/v3/role_assignments?group.id=f7a60598a30041ea9616052223d1ba2d&scope.project.id=4ea26c15401c49d28fd797d4ee85aa6c&role.id=47aca6aac2934a5cb816872abfc94994 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:23.123042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc82e84-6ca3-48d1-979d-cad46cf2ccfd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.124097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc82e84-6ca3-48d1-979d-cad46cf2ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.124580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc82e84-6ca3-48d1-979d-cad46cf2ccfd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.126498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 234/456] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:23.127623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6250572-30fb-4dec-826e-72aa63d18590 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.128059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6250572-30fb-4dec-826e-72aa63d18590 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.128199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6250572-30fb-4dec-826e-72aa63d18590 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.129948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 223/457] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:00:23.132793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.133084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.133373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.135207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1d43-4d6f-46dd-95e7-472e075bf38b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.135703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1d43-4d6f-46dd-95e7-472e075bf38b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.136334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1d43-4d6f-46dd-95e7-472e075bf38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.152714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:23.154425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:23.159388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eb4d1d43-4d6f-46dd-95e7-472e075bf38b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:23.160600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eb4d1d43-4d6f-46dd-95e7-472e075bf38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:23.232139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.245803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 224/458] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:23.247078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5fb7-f1de-43ce-a75e-b0a8cbec8cda None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.247475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5fb7-f1de-43ce-a75e-b0a8cbec8cda None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.247758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5fb7-f1de-43ce-a75e-b0a8cbec8cda None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.249990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 225/459] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:23.256359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.256644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.256927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.270079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 235/460] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:23.273253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.284268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:23.285840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:23.357217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.357664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 226/461] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:23.358672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.389216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e7589-b5db-4a79-880d-fd247d557983 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.434704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.435811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.436308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.436782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.447162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:23.448917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:23.461969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e03798-2080-49bf-8930-82aeff2836bb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:23.465226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:23.466041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:23.466903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:23.466903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:23.466903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:23.466903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:23.469071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 236/462] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:23.470060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.485121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.503584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.504469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.505003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.505413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.516689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:23.518420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:23.534735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:23.535605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:23.536468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:23.536468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:23.536468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:23.536468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:23.538490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 227/463] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:23.539455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.576638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0c27e-6f58-4ec4-917e-bacdc8e58f24 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.612342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.612993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.613272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.613548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.617383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.632398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:23.634924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:23.655710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:23.656642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:23.657484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:23.657484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:23.657484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:23.657484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:23.660078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 237/464] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:23.661088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.667809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26abbe-f19a-4275-b9a2-5315f0b11a3c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.711860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.712655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.713034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.713402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.714206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:23.714614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:23.716229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:23.734576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 228/465] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:23.734576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.766579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.794631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8c546-ef4a-4fd9-ad88-dbf5cf6e7c2d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.829370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.829989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.830271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.830549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.831300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:23.831613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:23.833032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:23.838237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115637af-c9e9-43e4-bfd6-00da17c2da6d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.848056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 238/466] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:23.848780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.875001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.875614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.875939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.876190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.876878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:23.877183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:23.878652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969be32-df1b-4ee0-8b9f-78c6619a4800 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:23.890559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 229/467] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:23.891823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0316445-2573-4088-95f5-e49d2a2bb02f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:23.892111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0316445-2573-4088-95f5-e49d2a2bb02f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:23.892386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0316445-2573-4088-95f5-e49d2a2bb02f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:23.893710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 230/468] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:23.896551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0316445-2573-4088-95f5-e49d2a2bb02f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:23.903212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.958701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52827a1-72fc-460e-94f7-b7e0fc637eba None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:23.992629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:23.999137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:23.999732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.000043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.000476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.001046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.002541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.005024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0316445-2573-4088-95f5-e49d2a2bb02f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:24.028086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 239/469] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:00:23 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:24.029285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:24.055717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09510691-947f-49ca-bc3b-96a098e525ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:24.056698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510691-947f-49ca-bc3b-96a098e525ac None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.057183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510691-947f-49ca-bc3b-96a098e525ac None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.057629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510691-947f-49ca-bc3b-96a098e525ac None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.058738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09510691-947f-49ca-bc3b-96a098e525ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.060316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09510691-947f-49ca-bc3b-96a098e525ac None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.086341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 231/470] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:00:23 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:24.087886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.088176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.088450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.111650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:24.113315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:24.221996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46838019-c7ae-4068-b179-46a1d2c4438c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:24.224413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:24.236209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 232/471] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:24.275161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:24.275846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:24.276739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.277237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.277753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.278907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.280722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.306551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 240/472] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:00:24 2022] POST /identity/v3/users => generated 248 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:24.379381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:24.408261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf32f1f-5adb-45c8-aa45-7865e126a2e4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:24.446704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:24.447403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.447693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.448000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.460789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.462297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-264f8d23-f667-4936-bc41-18296f8a41ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:24.478586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:24.479503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:24.480661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:24.480661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:24.480661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:24.480661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:24.480661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 233/473] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:24.488906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:24.577226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b4e8-541b-4d56-9205-c250a09bb3dd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.577526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b4e8-541b-4d56-9205-c250a09bb3dd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.577715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b4e8-541b-4d56-9205-c250a09bb3dd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.579258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 234/474] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:00:24.584596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.584883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.585161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.608200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:24.609395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:24.625422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3dc9-ba4e-48a5-87e8-798b6205ec05 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:24.680427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:24.681066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.681346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.681617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.682325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:24.682630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.684035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.705632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 241/475] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:24.716900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:24.721647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 235/476] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:24.746647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:24.759235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:24.845358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:24.904040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1653b8-8c63-4b23-9c45-4a8786cbf799 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:24.907221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7187bc-d6eb-41ec-925d-991c20207414 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:24.963957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:24.964597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.964879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.965155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.965848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.967351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.969417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:24.970057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:24.970337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:24.970614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:24.971342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:24.972750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:24.984190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.sql.core [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 05:00:24.984190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 05:00:24.984190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [parameters: {'id': 'df487a6acf6f443383d1a98d5b62f995', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 05:00:24.984190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80378) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 05:00:25.004344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 242/477] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:00:24 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Aug 05 05:00:25.007362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 05:00:25.008313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application yield resource Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise self.value Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application session.commit() Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 05:00:25.009222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.flush() Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application compat.raise_( Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 05:00:25.010124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 05:00:25.010997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 05:00:25.011817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [parameters: {'id': 'df487a6acf6f443383d1a98d5b62f995', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:25.012596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 05:00:25.013455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:25.014367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 236/478] 10.222.0.30 () {60 vars in 1131 bytes} [Fri Aug 5 05:00:24 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:25.017452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:25.103876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:25.146502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9c36b-7104-4a22-8950-cbb3a9bc3994 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:25.193116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bc900-0908-4dc2-ae33-d0707f277b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:25.194159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc900-0908-4dc2-ae33-d0707f277b2e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:25.194635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc900-0908-4dc2-ae33-d0707f277b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:25.195101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc900-0908-4dc2-ae33-d0707f277b2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:25.215318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bc900-0908-4dc2-ae33-d0707f277b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a48bc900-0908-4dc2-ae33-d0707f277b2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:25.221391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:25.222743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 05:00:25.223535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:25.226423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 243/479] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:25.235565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:25.342697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59d40e-56af-4f3b-9f03-48c1a70a0528 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:25.366560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:25.380183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:25.380820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:25.381321 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:25.381603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:25.382407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:25.382708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:25.384101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4fa0-8ebf-44c6-870c-4571eeeecdf9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:25.393639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 237/480] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:26.616270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ade20-e8d8-4ccc-a058-412092fc1db4 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.617321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ade20-e8d8-4ccc-a058-412092fc1db4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.617799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ade20-e8d8-4ccc-a058-412092fc1db4 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:26.621178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cda7d-bcf9-46aa-876d-3cb88c570b13 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.621881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 244/481] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:26.622430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cda7d-bcf9-46aa-876d-3cb88c570b13 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.622928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cda7d-bcf9-46aa-876d-3cb88c570b13 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:26.624594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f43e-ce31-48ff-9528-de63767dcac8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.625071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f43e-ce31-48ff-9528-de63767dcac8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.626772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f43e-ce31-48ff-9528-de63767dcac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:26.629870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 238/482] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:26.631374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6d9b2-4031-4f42-91dc-5ecd76f5564b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.631901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6d9b2-4031-4f42-91dc-5ecd76f5564b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.632368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6d9b2-4031-4f42-91dc-5ecd76f5564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:26.662748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-95f9f43e-ce31-48ff-9528-de63767dcac8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:26.664295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-95f9f43e-ce31-48ff-9528-de63767dcac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:26.667323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-67c6d9b2-4031-4f42-91dc-5ecd76f5564b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:26.668729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-67c6d9b2-4031-4f42-91dc-5ecd76f5564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:26.772806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 239/483] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:26.773976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c290022-62f2-4584-af17-9ebc85988eb3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.774271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c290022-62f2-4584-af17-9ebc85988eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.774551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c290022-62f2-4584-af17-9ebc85988eb3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:26.775331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 245/484] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:26.775982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 240/485] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:26.781296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.787868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.787868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:26.808292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c290022-62f2-4584-af17-9ebc85988eb3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:26.812791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:26.813980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:26.872181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:26.881054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 246/486] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:26.886551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:26.939262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c290022-62f2-4584-af17-9ebc85988eb3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:26.954298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:26.981134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14940f-2d81-4133-abe7-13c22fa85559 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:26.982140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:26.983259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:26.984236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:26.984623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.008282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:27.021842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:27.022795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 241/487] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:27.023585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.035367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.036061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.036342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.036616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.047377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:27.052693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:27.053593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:27.054390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 247/488] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:27.056413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.113836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.128367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb733e-dfd7-4da4-96d5-ccc791bc6dca None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.159805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f76d7-1507-4d29-b70f-9fe0c381a235 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.167322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.168013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.168323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.168872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.180387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:27.185620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:27.186507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:27.187588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 242/489] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:27.188449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.200128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.201213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.201683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.202140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.203047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:27.203536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.205138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.218462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 248/490] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:27.219731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.225057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.296872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317d210-4e9c-4a90-b6f8-269ad97981bf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.321666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb93ad-94da-4f9a-b295-5d0bc960d4f5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.334141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.334766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.335088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.335370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.336243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:27.336547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.338090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.339825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.350843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 243/491] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:27.352053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.358909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.359537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.359843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.360116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.360830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:27.361133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.362499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.370716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 249/492] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:27.371421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.458076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.465482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b640ca3-55fc-454d-853f-cfbc65ddd3c9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.470803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30139310-9491-4654-8c84-9450b95e4ed1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.503606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.504598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.504972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.505248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.505526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.505803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.506072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.506586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.517235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.518712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.519034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.520854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:27.533231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:27.534090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:27.534945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:27.534945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:27.534945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:27.534945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:27.534945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 250/493] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:27.535641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:27.539284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:27.540122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:27.540959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:27.540959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:27.540959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:27.540959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:27.543121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 244/494] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:27.544194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.571536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.641052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0057c-7305-411f-a315-62bb0c069aec None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.658880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71364288-54d4-430a-b487-065e2bd64897 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.675494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.676178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.676458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.676886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.688248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.689728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.694424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.695062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.695344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.695616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.696346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:27.696654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.698032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01ba50-cec0-4deb-8252-ca14e575803e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:27.706039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:27.707181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:27.708127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:27.708127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:27.708127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:27.708127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:27.709092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 251/495] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:27.709884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.714425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 245/496] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:27.715685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9edb2-1c34-448c-850d-a26a25c7b860 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.716016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9edb2-1c34-448c-850d-a26a25c7b860 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.716332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9edb2-1c34-448c-850d-a26a25c7b860 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.717713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 246/497] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:27.722170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9edb2-1c34-448c-850d-a26a25c7b860 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.733206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.805278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.824997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84408108-bcc0-44ab-98af-ce673f4a6efd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.827322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9edb2-1c34-448c-850d-a26a25c7b860 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:27.859188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.859848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.860131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.860405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.861109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:27.861409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.862470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:27.862783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.863092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.863375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.863647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.864426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:27.864732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:27.866122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867d905-477d-4099-b379-a7ebf4c85396 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:27.879532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 252/498] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:27.879532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.886583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 247/499] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:27.886583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:27.886583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:27.886583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:27.901739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:27.902979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:27.968809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:27.979792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 248/500] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:27.979792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:27.985647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d9151-0b54-4c27-9e45-30b86bcae625 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.023025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.023673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.023996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.024272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.031303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.032784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:28.045585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:28.046460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:28.047332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 253/501] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:28.048464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.050157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.088414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d1bb6-12d7-4faf-9c0c-9ede30200606 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.121118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.121759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.122040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.122341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.129184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.130633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:28.143294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:28.144211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:28.145104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 249/502] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:00:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:28.145809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.151554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56283f-ffb7-4eb2-b583-236cfda22328 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.185946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.186576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.186855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.187129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.195284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.196775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.204367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c803b291-1687-4490-bf3d-3213ab851530 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:28.208267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:28.209175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:28.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 254/503] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:28.210588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5865fa4-1f03-4023-9e48-2bb100e639a5 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.210959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5865fa4-1f03-4023-9e48-2bb100e639a5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.211053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5865fa4-1f03-4023-9e48-2bb100e639a5 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.212519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 255/504] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:28.216655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5865fa4-1f03-4023-9e48-2bb100e639a5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.248390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520918-ce69-4ec5-8177-40d3d39cb338 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.277502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.280625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.281273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.281558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.281846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.292059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:28.297387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:28.298237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:28.299015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 250/505] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:28.299814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.321089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5865fa4-1f03-4023-9e48-2bb100e639a5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.352717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.353336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.353623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.353918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.354707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:28.355000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.356494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.391705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 256/506] 10.222.0.30 () {56 vars in 1137 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:28.392553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.400911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabe821-8bff-495b-ba8b-8b3653906307 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.418494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.435911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.436554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.436832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.437102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.437894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:28.438201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.439642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab89671-500a-4623-abca-039c387f9f14 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.484430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 251/507] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:28.485690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.485994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.486263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.491278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e734f-d3ac-4798-9808-96e6a8e70a16 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.504057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:28.505189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:28.518311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.535977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.536614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.536897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.537173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.538132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:28.538601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.540902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.560997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 257/508] 10.222.0.30 () {56 vars in 1137 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:28.561950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.577384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 252/509] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:28.578151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.661731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296f105-e71f-452d-a599-a4b18c436093 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.673089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedc8e7-b055-4bcc-aa2f-79813b022af9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.680495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.694798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.695426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.695708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.696010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.696730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:28.697039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.698427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.706304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 258/510] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:28.707082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.708297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.708921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.709200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.709474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.747684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.784439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.785847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.806008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2ca8c-41b0-45db-be73-014c7337f38e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:28.842511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:28.843174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:28.843454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:28.843724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:28.889778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 253/511] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:28.890641 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:28.899327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.926235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:28.928121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:28.967047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:28.990936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54000a19-fca1-4951-88c9-5914db822e87 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.025334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 259/512] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:29.026268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.026679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.027313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.027601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.027911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.094856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.096311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.103795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.121278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375992fe-808e-4f73-b735-1b13bbf91c73 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.155442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.156107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.156382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.156648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.164930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.167101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.168684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:29.183790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:29.184701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:29.185588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:29.185588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:29.185588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.185588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:29.185588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 260/513] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:29.186076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.201681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 254/514] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:29.203025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.286010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380d2cf-7a2c-4f39-9c6d-a925a00387a6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.300697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed085-c1ec-4dc3-84bc-fae5d746236c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.308116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.320652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584674-226f-4134-8c12-67568fcee903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.321302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584674-226f-4134-8c12-67568fcee903 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.321584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584674-226f-4134-8c12-67568fcee903 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.321853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584674-226f-4134-8c12-67568fcee903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.332431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03584674-226f-4134-8c12-67568fcee903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.333888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03584674-226f-4134-8c12-67568fcee903 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.334392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.335059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.335338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.335611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.345390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.346786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-03584674-226f-4134-8c12-67568fcee903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:29.348466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:29.349348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:29.350162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:29.350162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:29.350162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.350162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:29.350162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 261/515] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:29.350776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584674-226f-4134-8c12-67568fcee903 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:29.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:29.365825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:29.366639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:29.366639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:29.366639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.366639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:29.366639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 255/516] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:29.367284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.396927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.450657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584674-226f-4134-8c12-67568fcee903 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.466219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b65fa-c37e-485d-b174-3d10d2548561 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.482749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.483434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.483706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.484019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.494218 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.495694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.499040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.499658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.499974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.500250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.500943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:29.501251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.502609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:29.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:29.510204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:29.511031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:29.511031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:29.511031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.511031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:29.511031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 262/517] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:29.511279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.518559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 256/518] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:29.519360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.541434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.609770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.615363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0801b-93f4-430c-8b0f-d258aada7ff2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.620384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288fe62-d874-413c-9165-1392a1655959 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.649541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.650010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.650282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.650548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.654261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75378609-66af-4273-a04d-9732e629657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.654889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75378609-66af-4273-a04d-9732e629657f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.655172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75378609-66af-4273-a04d-9732e629657f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.655447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75378609-66af-4273-a04d-9732e629657f None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.656204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75378609-66af-4273-a04d-9732e629657f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:29.656516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75378609-66af-4273-a04d-9732e629657f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.657925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75378609-66af-4273-a04d-9732e629657f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.660759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.662200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.669313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 257/519] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:29.670098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75378609-66af-4273-a04d-9732e629657f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:29.675980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:29.676839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:29.678048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:29.678048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:29.678048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:29.678048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:29.678048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 263/520] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:29.678311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.748867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.771702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75378609-66af-4273-a04d-9732e629657f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.778153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2f69-75ef-498b-ba17-81c2b167c831 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.804682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.805304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.805587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.805860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.806548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:29.806857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.808288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.812466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.813071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.813349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.813617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.814294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:29.814594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.815975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.819714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 258/521] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:29.820502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.829675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 264/522] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:29.830842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.841596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.921103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46406a94-6d75-4788-863f-f1a2fd853f7a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.928832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0d3de-f309-48f7-8845-a0040b3aaea9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:29.954670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.955353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.955633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.955936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.956593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.958029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.961145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:29.961781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:29.962059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:29.962329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:29.967699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:29.969532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:29.971012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:29.979997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 259/523] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:00:29 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:29.980796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:29.982712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:29.983634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 265/524] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:29.984930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.065162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.101995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611031c-d0cf-429b-bc28-5fa3397b2f01 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.111331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcc4c1-2a2d-43ef-8963-c64e571767bd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.139346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.140036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.140335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.141260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.148000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.149469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.149871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.150505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.150783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.151040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.151726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:30.152144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.154041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:30.161785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:30.162595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:30.163502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 266/525] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:30.163995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.171279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 260/526] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:30.172084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.241175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.285863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81da776-f0cc-41ab-9218-6dc46eaf95d8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.289928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a51d6-363c-48dc-bda1-c16fa18a9e01 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.316173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.326214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.326939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.327136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.327264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.327483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.327736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.328061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.328336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.329130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:30.329610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.331363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.334614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.335776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.353309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 261/527] 10.222.0.30 () {56 vars in 1139 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:30.353502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:30.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:30.355192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:30.355602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.355680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 267/528] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:30.355680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.459462 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953191a1-7971-4c78-b584-7bc835642f59 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.460895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e275c-0578-4f36-8782-706bc59e391d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.473458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.495361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.496039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.496322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.496654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.498106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.498731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.499016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.499286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.500092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:30.500399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.501839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.504320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.506315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:30.519193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:30.520113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:30.520959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:30.521353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 262/529] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:30.521353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.523225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 268/530] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:30.524180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.578968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.636457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bcd82-123b-4918-9956-3bfdb66d2d3a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.642948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b075136-9b6e-467e-b4d6-8e1be742abb8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.671465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.672700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.672978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.673253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.674001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:30.674308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.675728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.677205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.678198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.678478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.678753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.698830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 269/531] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:30.698830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.727028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.755430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.756943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.800967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87746ede-0c92-4c6d-b7c2-69a6b44d7721 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.810438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.836052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.836677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.836954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.837221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.838004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=a88bb908a0104cc589b19095091a8a53) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:30.838310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.839730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:30.855810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 270/532] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/role_assignments?user.id=a88bb908a0104cc589b19095091a8a53&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:30.855810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.880503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 263/533] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:30.881740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:30.955980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dcdae-482f-46b8-97f1-faecee7c5d80 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.990059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:30.993194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:30.994041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:30.994488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:30.994915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:30.995903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:30.996604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:30.999265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebb6b5-9110-486d-8960-f80aec618ed7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:30.999559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.014226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 271/534] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:31.015790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:31.055236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:31.055998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:31.056343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:31.056744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:31.057739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=5134c03bb8854cd4bee58644e6eacc51) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:31.058245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:31.058967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:31.060660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.066957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 264/535] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:30 2022] GET /identity/v3/role_assignments?user.id=5134c03bb8854cd4bee58644e6eacc51&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:31.067986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:31.154935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588a5d-3fe6-4fad-afa9-6e1c451c1cc0 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:31.188784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedf7d-98c7-4a00-abba-d51c459cc119 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:31.192286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:31.192867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:31.193237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:31.193603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:31.205316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:31.207113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:31.240975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:31.241853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:31.242685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:31.242685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:31.242685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:31.242685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:31.242685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 272/536] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:31.252721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:31.268941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:31.269584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:31.269864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:31.270136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:31.270936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=ea0805461cd3492fa5e2ebbea407e351) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:31.271246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:31.272780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.289782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:31.295356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 265/537] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:31 2022] GET /identity/v3/role_assignments?user.id=ea0805461cd3492fa5e2ebbea407e351&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:31.362658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:31.380401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ede11-4254-4dfc-90c7-41154fd6e031 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:31.416998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:31.417633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:31.417913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:31.418187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:31.418914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:31.419238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:31.421069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.434491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 273/538] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:31.441136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:31.585691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994e2eb-b616-44ae-a3bc-f69baad0c9a0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:31.630527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:31.631509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:31.631998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:31.632426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:31.648908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:31.650406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:31.666065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:31.666956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:31.667807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 266/539] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:31.673715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:31.827325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d45200-4d19-44c3-98d8-802f946f2d45 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:31.881526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:31.882090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:31.882370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:31.882641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:31.883405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:31.883711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:31.885266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680eca1-5b1d-47a4-8596-d1601a5715e4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:31.902653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 274/540] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:31.913245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:31.985243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:32.049999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe82aa-a77d-4d52-8912-12e04b73e8a4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:32.100481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:32.101439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:32.101909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:32.102366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:32.103331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=cfb671f152cd4c40825a81eec15af784) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:32.103878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:32.105504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26b945-af92-4107-a6aa-4cb4351cfa4e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:32.116548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 267/541] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:31 2022] GET /identity/v3/role_assignments?user.id=cfb671f152cd4c40825a81eec15af784&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:32.148704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819be3f-56f7-46c9-8a64-931d5a882a7f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:32.148895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819be3f-56f7-46c9-8a64-931d5a882a7f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:32.149183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819be3f-56f7-46c9-8a64-931d5a882a7f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:32.150546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 275/542] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:32.163881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:32.164510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:32.164972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:32.193371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:32.194994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:32.288011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 268/543] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:32.332364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819be3f-56f7-46c9-8a64-931d5a882a7f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:32.341060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:32.479153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819be3f-56f7-46c9-8a64-931d5a882a7f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:32.523654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:32.524734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:32.525156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:32.525563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:32.543601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:32.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:32.554325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:32.555321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:32.557310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 276/544] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:32.563440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:32.664342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:32.717390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd240-b6c9-4b8b-8aad-079eb0c9bbea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:32.763522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:32.764109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:32.764391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:32.764663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:32.765395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:32.765698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:32.767080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:32.779655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 269/545] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:32.785331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:32.857113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:32.913086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07463287-3c0e-40a1-b1b4-9ee324c68824 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:32.952543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:32.953562 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:32.954036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:32.954460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:32.970369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:32.972719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:32.991062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:32.991968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:32.992792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:32.992792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:32.992792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:32.992792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:32.993512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 277/546] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:33.000647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:33.132134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf60b9d-3658-4c84-bffe-bc23e58ab01b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:33.153525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:33.178211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:33.178898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.179187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.179465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.180194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:33.180507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:33.181897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c81b3e-b62d-4d85-b1e4-f80b19673b84 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:33.199858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 270/547] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:33.205650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:33.336739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf31a-dae9-4572-bd10-48e4c44fe630 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:33.359070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:33.367087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d794f-5a78-4070-b999-649e8734b42b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.367523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d794f-5a78-4070-b999-649e8734b42b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.367967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d794f-5a78-4070-b999-649e8734b42b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.370439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 271/548] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:33.377037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.377315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.377596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.392213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:33.393329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.393818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.394295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.405153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:33.407673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:33.416792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:33.418005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:33.420767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:33.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:33.423256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:33.426067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 278/549] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:33.432361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:33.544498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 272/550] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:33.580242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe921e-8657-48a9-9f72-d50c379c5e23 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:33.589327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:33.644649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:33.645684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.646960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.647418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.648781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:33.653717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:33.655386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:33.696385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 279/551] 10.222.0.30 () {56 vars in 1135 bytes} [Fri Aug 5 05:00:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:33.703128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:33.743643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:33.791285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ee0c-c89b-42ae-9eb3-6ed0b0825ff5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:33.799442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:33.818765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8508acf-93fb-4aec-bb52-02097569939d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:33.833079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be7161-c947-4161-9357-732476e48995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:33.834126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be7161-c947-4161-9357-732476e48995 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.834677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be7161-c947-4161-9357-732476e48995 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.834960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be7161-c947-4161-9357-732476e48995 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.849768 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be7161-c947-4161-9357-732476e48995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:33.853139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1a442-589b-458c-b612-21a1e76b18a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:33.853765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1a442-589b-458c-b612-21a1e76b18a1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.854041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1a442-589b-458c-b612-21a1e76b18a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.854311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1a442-589b-458c-b612-21a1e76b18a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b5be7161-c947-4161-9357-732476e48995 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:33.854661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:33.855658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:33.856717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 273/552] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:33.857417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635327-4b75-401c-a288-b685658e8d99 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.857417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635327-4b75-401c-a288-b685658e8d99 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.858084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635327-4b75-401c-a288-b685658e8d99 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.858936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 274/553] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:33.863846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda289c-3e31-4b17-8987-dc636579754a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:33.863846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda289c-3e31-4b17-8987-dc636579754a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:33.863846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda289c-3e31-4b17-8987-dc636579754a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:33.867414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 275/554] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:33.868379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda289c-3e31-4b17-8987-dc636579754a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:33.934923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1a442-589b-458c-b612-21a1e76b18a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:33.936436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1a442-589b-458c-b612-21a1e76b18a1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:33.970131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:33.974169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda289c-3e31-4b17-8987-dc636579754a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.014196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.014881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.015112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.015389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.016138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:34.016445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.017833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3877-2ce5-4568-81be-6aa5c6f01bbb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.030977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 276/555] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:34.030977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.030977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.030977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.045123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:34.049944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 280/556] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:34.050089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:34.050169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.050169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.050169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.065693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:34.067333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:34.072192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.117245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 277/557] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:34.117914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.152278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 281/558] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:34.153185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.221007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.232372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab3e0-34f2-41bc-97cf-693a5609eec3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.261639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c0c22-5998-4ea3-8332-b41965830894 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.269112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.270010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.270508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.271024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.283429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.284936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.295411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.296386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.296887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.297356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:34.299813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:34.300753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:34.301614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:34.301614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:34.301614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:34.301614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:34.305914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 278/559] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:34.306779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.312193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.313827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.319595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:34.331641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:34.333042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:34.334500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:34.334500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:34.334500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:34.334500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:34.334500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 282/560] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:34.334500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.416207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ec9c1-db2d-4390-b933-6a6db0e3fdec None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.442448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eba3d-9eac-4eae-9c6b-5d2d6de5dd31 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.453142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.454099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.454661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.454942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.459568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.468688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:34.475675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:34.476618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:34.477547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 279/561] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:34.478266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.482844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.483502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.483803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.484082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.494399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:34.499744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:34.500677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:34.501533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 283/562] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:34.502313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.575643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.616227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd4efb-9b1a-4df6-8b0e-f3e49db90442 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.620357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5408349-4eaf-4f79-8f7e-bf449b6227f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.651296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.651949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.652227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.652504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.653198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:34.653580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.654899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.656009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.656636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.656920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.657199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.657892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:34.658201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.659566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.668280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 284/563] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:34.669562 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.675864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 280/564] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:34.676760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.707785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.771947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c61cb-b8d2-4cd8-95bf-3eea1b0dc803 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.778618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c070a5d-ec31-4cf4-aca9-c258151a6a9c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.805806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.806446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.806727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.807003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.807716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:34.808045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.808971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.809418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.814633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.815232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.815526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.815824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.816541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:34.816846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.817664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 285/565] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:34.818219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.818322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.829476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 281/566] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:34.830236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.912338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:34.916169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195fbe-1ee9-4fb2-85ce-901a58405431 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.927130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee02d3-2759-41f0-9651-9b28af0fd2b7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:34.950405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.951046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.951325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.951623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.958169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.959621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.960210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:34.960849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.961125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.961416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.968705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:34.970134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-828c4065-4b27-41f8-b42f-36ed530c8b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:34.970866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:34.971720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:34.972634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 286/567] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:34.973153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fe295-f3a2-4fd3-8001-b66bd83bee8d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:34.973431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fe295-f3a2-4fd3-8001-b66bd83bee8d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:34.973709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fe295-f3a2-4fd3-8001-b66bd83bee8d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:34.974955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 287/568] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:34.975693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fe295-f3a2-4fd3-8001-b66bd83bee8d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:34.984989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:34.986163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 282/569] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:34.987362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.043039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.087966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fe295-f3a2-4fd3-8001-b66bd83bee8d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.090493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f737b-7be6-4c6a-aee5-7f1f79ba36f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.119894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.120533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.120806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.121076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.123274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.123950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.124230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.124504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.131637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.133109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.134669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.136129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:35.147244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:35.148158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:35.149045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:35.149045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:35.149045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:35.149045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:35.149045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 288/570] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:35.149325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-97b8844a-b907-49d4-8b81-f368a97589bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:35.149807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:35.150831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:35.152058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:35.152058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:35.152058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:35.152058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:35.152058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 283/571] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:35.152390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db4927-8e45-47bc-9a45-825925021dca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.152620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db4927-8e45-47bc-9a45-825925021dca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.152895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db4927-8e45-47bc-9a45-825925021dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.169845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.171997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-50db4927-8e45-47bc-9a45-825925021dca None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:35.173194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-50db4927-8e45-47bc-9a45-825925021dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:35.238917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 284/572] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:35.239737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db4927-8e45-47bc-9a45-825925021dca None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.260471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370113d-1196-4809-835d-f41938554eb9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.273922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.294450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.295005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.295283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.295561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.296340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:35.296649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.298116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.311007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 289/573] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:35.311790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.342005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db4927-8e45-47bc-9a45-825925021dca None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.375211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.375820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.376096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.376367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.377152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:35.377453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.378888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.379546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.395617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 285/574] 10.222.0.30 () {56 vars in 1137 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:35.396374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.415629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b2e44-7377-41ad-899a-3f42a71b8c64 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.449526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.450144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.450481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.450756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.451440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:35.451755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.453176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.464382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 290/575] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:35.465157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.490605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.499783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584013b1-9eb7-4ce3-9430-38e03b1c0087 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.532103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.532717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.532999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.533259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.533948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:35.534270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.535653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.546846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 286/576] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:35.547937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.563181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13d725-c8d6-4688-ba39-7710826afde6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.593964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.600080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.600727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.601002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.601276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.611126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:35.616485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:35.617602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:35.618446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 291/577] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:35.619118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.650525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09336-57f5-4080-83e0-185018665cfc None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.683433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.684075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.684354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.684643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.685441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=127c0f72286541159eac49c8f958f816) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:35.685743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.687171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.703891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 287/578] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/role_assignments?user.id=127c0f72286541159eac49c8f958f816&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:35.703891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.720330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0f4e-fc61-4480-b5ad-94ba6238d9d1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.729657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.758800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.759466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.759744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.760047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.772772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.774260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:35.789888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:35.790974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:35.792062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:35.792062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:35.792062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:35.792062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:35.796081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 292/579] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:35.797152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.828026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30103-61b3-4782-8650-1fd44fbfc2f3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.844610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.864162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.865096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.865379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.865651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.873446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.874892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:35.888838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:35.889808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:35.890762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:35.892593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 288/580] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:35.893488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:35.909543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44688e5-54f4-4fab-b3c2-bc76cb0bc711 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:35.945431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:35.946821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:35.947240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:35.947654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:35.955966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:35.957662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:35.969799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:35.970940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:00:35.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:35.973253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:35.975091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 293/581] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:35.976035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.003306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503a926-deb8-45d8-9059-c9305ff46b2b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.040103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.040726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.041038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.041278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.042015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.042299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.043686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.052035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 289/582] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.052950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.094996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacdfa2-9136-4cc4-9410-967760a3ed1d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.096710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.132226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.132948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.133248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.133503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.134199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.134500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.135920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.148095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 294/583] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.148802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.165171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8380c64-b074-45b8-865f-b9e55978a9a5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.199887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.200618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.200894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.201172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.201967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.202276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.203736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.212575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.224207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 290/584] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.224660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.257471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1deb8-da8e-46bc-beb6-7eff52082993 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.292567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.293189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.293538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.293729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.294630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.295494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.296398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.314044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 295/585] 10.222.0.30 () {56 vars in 1137 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.315107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.330574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d92d9-1ccb-4f5f-a0a6-0917e3e3082c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.337103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.368444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.369088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.369369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.369958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.380367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.381877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:36.396485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:36.397406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:36.398292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:36.398292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:36.398292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:36.398292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:36.398292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 291/586] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:36.398995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.423494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203006e-e080-4796-a8e6-605700dd1713 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.450657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.458222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.458866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.459210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.459466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.468437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.470428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:36.484553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:36.485435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:36.486307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 296/587] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:36.492281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.501842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22a12-863c-4b40-891c-0e06d33e8581 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.536704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.537347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.537633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.537899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.548678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.550325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:36.565033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:36.565951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:36.566825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:36.566825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:36.566825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:36.566825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:36.566825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 292/588] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:36.566825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.579339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.600629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679eedb-7137-4534-8279-63bbb1d8aaf1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.633184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.633829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.634105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.634382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.645504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.647024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:36.661569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:36.662456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:36.663338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:36.663338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:36.663338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:36.663338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:36.663338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 297/589] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:36.663630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.681003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.685049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470638-7a20-4899-9471-b69fe3c7db9b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.725530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.726154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.726437 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.726713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.727397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.727702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.729162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.746856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 293/590] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.746856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.770116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0a7e8-635e-45d2-bf74-54d757b14fcc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.801910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.809046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.809753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.810002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.810281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.811034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.811339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.812801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.829889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 298/591] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.829889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.857755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da260ffd-e7c3-4f7e-97b0-854f3c9c4def None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.891055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.891671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.891978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.892249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.892935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.893254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.894609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.906660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 294/592] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.907841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:36.932080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:36.935956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9da05-5dd1-4782-afd9-1b29595695a2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:36.971786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:36.972446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:36.972728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:36.973001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:36.973692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:36.973995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:36.975370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:36.987570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 299/593] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:36.992705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.019471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee59db-0935-4efd-beba-a8b53720dc26 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.048516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.055036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.055694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.056008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.056286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.063921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.065374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:37.076485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:37.077540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:37.078602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 295/594] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:37.082934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.107823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55932638-7f9b-48ef-9857-f555f34632a3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.145575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.146248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.146736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.146846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/a88bb908a0104cc589b19095091a8a53/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.162149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=a88bb908a0104cc589b19095091a8a53, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.163599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.170597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.187802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 300/595] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:36 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/a88bb908a0104cc589b19095091a8a53/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:37.187802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.204353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de63cb-bd79-4938-81d8-b77c9a1fc08b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.251343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.252303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.252589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.252867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.261414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.263182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:37.278578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:37.279556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:37.280471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 296/596] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:37.280969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.325624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d3f58-d572-43e0-bb29-1e21c772b0ff None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.327657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.371352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.371966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.372374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.372650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.381012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.382634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:37.395122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:37.396012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:37.396912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 301/597] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:37.397466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.408425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311c4ec-ffb8-429e-b078-9a0630a6b515 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.455200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.456937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.457403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.457875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.458902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:37.459937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.462087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.475694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.488639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 297/598] 10.222.0.30 () {56 vars in 1139 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:37.493974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.517130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa521efa-b4a2-4c19-9eac-b83712d01228 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.564903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.566180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.567506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.568019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.568979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:37.569503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.571172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.591059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 302/599] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:37.592251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.600209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8c82f-377c-47f3-9fbc-11cbcf85b669 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.621775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.635470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.636364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.636476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.636668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.637491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:37.638081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.639341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.652183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 298/600] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:37.653132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.697543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1901c-adb3-472b-8194-2fbbe401c8c4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.711632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.740680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.741574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.741999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.742413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.754385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.756074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.756365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0b358-b24a-4a3e-bc3e-6695d23e198a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:37.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:37.774503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:37.775641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:37.775641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:37.775641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:37.775641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:37.782924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 303/601] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:37.783984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.796956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585534b5-6154-4594-8ebd-1b106d5f10e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.797951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585534b5-6154-4594-8ebd-1b106d5f10e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.798234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585534b5-6154-4594-8ebd-1b106d5f10e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.798522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585534b5-6154-4594-8ebd-1b106d5f10e8 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/5134c03bb8854cd4bee58644e6eacc51/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.813610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585534b5-6154-4594-8ebd-1b106d5f10e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=5134c03bb8854cd4bee58644e6eacc51, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.815054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585534b5-6154-4594-8ebd-1b106d5f10e8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.830650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 299/602] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:37 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/5134c03bb8854cd4bee58644e6eacc51/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:37.832047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e36eca-62cd-450b-b300-93a59a77f620 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.832425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e36eca-62cd-450b-b300-93a59a77f620 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.832790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e36eca-62cd-450b-b300-93a59a77f620 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.835853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 300/603] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:37.835853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e36eca-62cd-450b-b300-93a59a77f620 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:37.877505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:37.911450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fcc42-29e6-4dab-b401-b6fb9ded6279 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.955084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e62491-bf20-46be-b41e-f1602d2153b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:37.956090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62491-bf20-46be-b41e-f1602d2153b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.956090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62491-bf20-46be-b41e-f1602d2153b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.956364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62491-bf20-46be-b41e-f1602d2153b3 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/ea0805461cd3492fa5e2ebbea407e351/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:37.973633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e62491-bf20-46be-b41e-f1602d2153b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=ea0805461cd3492fa5e2ebbea407e351, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:37.975286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e62491-bf20-46be-b41e-f1602d2153b3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:37.991910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e36eca-62cd-450b-b300-93a59a77f620 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:37.997020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 304/604] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:37 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/ea0805461cd3492fa5e2ebbea407e351/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:37.998664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:37.999450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:37.999450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.017256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:38.025855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:38.027144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:38.052864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.053877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.054350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.054912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.056042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:38.057085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.058789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.073037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 301/605] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:38.079185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.143217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 305/606] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:38.167153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.190021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca331e-17fb-499c-9ec7-788944e15789 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.226561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.227708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.228230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.228698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:38.229179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.237419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.239155 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:38.251724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:38.252743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:38.253610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:38.256689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 302/607] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:38.261894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.286058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634da93-7a39-4492-80cf-3cc6dc7d1a4b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.292607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:38.325349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.326011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.326409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.326670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.338167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:38.343698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:38.344646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:38.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 306/608] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:38.351437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.374302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a17789-77b5-4bc3-984b-882016e63412 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.410602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.411540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.412017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.412456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.413494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:38.413969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.415604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.430227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 303/609] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:38.449472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.476782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84deaa3-5e35-47d1-a278-488760287795 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.493839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:38.527185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.528097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.528529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.529123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.529832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:38.530253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.531886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.541441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 307/610] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:38.547248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.560682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:38.584153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f981b22-0996-47c8-951e-9f3ae6feb028 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.640524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.641203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.641546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.641820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/cfb671f152cd4c40825a81eec15af784/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.658958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ded9f-7bef-4a64-b3c0-cd7cb1e5d377 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.673268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=cfb671f152cd4c40825a81eec15af784, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.674675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.698403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 304/611] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:38 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/cfb671f152cd4c40825a81eec15af784/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:00:38.698596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.700105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.700105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.700105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.719393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.725028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:38.749995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:38.750955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:38.751931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:38.751931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:38.751931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:38.751931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:38.751931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 308/612] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:38.758491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.897436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63f5f6-6d8b-4640-981c-645dc150226d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:38.935080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:38.936107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:38.936646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:38.937117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:38.938004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:38.938577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:38.940183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:38.947558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:38.954029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 305/613] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:38.960826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:38.992837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:39.090599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9a8d-ea2e-412b-839e-e47cb6cd11ac None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:39.137926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:39.138638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.138918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.139191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:39.152805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:39.154310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:39.168376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:39.169412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:39.170749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 309/614] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:39.175673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:39.287816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b530c5-78d5-432a-9c26-9737c6fe2f6a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:39.326189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:39.326819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.327095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.327365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:39.328205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:39.328506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:39.329969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:39.348524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 306/615] 10.222.0.30 () {56 vars in 1135 bytes} [Fri Aug 5 05:00:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:39.354143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:39.462897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:39.476587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafe56f-147d-40a2-993e-3fe22d6954a6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:39.542026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:39.542735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.543171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.543585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:39.559119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:39.560760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5f9084c9-236a-4b04-974a-09d805eba587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:39.581170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:39.582066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:39.583545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:39.583545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:39.583545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:39.583545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:39.583545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 310/616] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:39.588049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:39.647029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:39.692879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19425a-8eae-490b-abb3-392e997d465a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:39.714748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c434f-5757-4d4c-9086-fe899214e926 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.715080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c434f-5757-4d4c-9086-fe899214e926 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.715371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c434f-5757-4d4c-9086-fe899214e926 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:39.717752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 311/617] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:00:39.729049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.729410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.729756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:39.747970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:39.748600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.748888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.749151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:39.749844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:39.750146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:39.751521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:39.760837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:39.762053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:39.767574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 307/618] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:39.772831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:39.910133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 312/619] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:39.917316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0de871-c66c-4416-9d4e-72bfaf0f2a1b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:39.937600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:39.956102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:39.990357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:39.991007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:39.993266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:39.993537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.017054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:40.019291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:40.032936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:40.033991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:40.034873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:40.037431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 308/620] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:40.043444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.096761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:40.139864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcfac6-d9d8-4fcb-a6b3-931b46ad5f43 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.201518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.202088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.202366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.202638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.219733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38eaf55-a066-44d8-b02b-41e8a776d3b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.222629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:40.231871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:40.232808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:40.233642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 313/621] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:40.240266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.290204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.290832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.291114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.291388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.292367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:40.292674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:40.296157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:40.319465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 309/622] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:40.325806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.387790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:40.422012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bcc76-d8e1-43ed-b9de-a6ff97e6f83e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.452827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:40.459188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.459756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.460264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.460814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.461814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:40.462567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:40.464005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0f6f3-ee62-4d89-801e-b5812465b689 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:40.465585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d3df3-fa89-471e-ad86-55549dd3c280 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.477383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 314/623] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:40.478783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820b8ca-d98b-4405-8227-5f56868f722d None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.479145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820b8ca-d98b-4405-8227-5f56868f722d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.479488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820b8ca-d98b-4405-8227-5f56868f722d None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.484955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 315/624] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:40.485865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820b8ca-d98b-4405-8227-5f56868f722d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.502768 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b13965-c7f1-4d0b-8805-f050d085bbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.503662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b13965-c7f1-4d0b-8805-f050d085bbd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.504180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b13965-c7f1-4d0b-8805-f050d085bbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.504587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b13965-c7f1-4d0b-8805-f050d085bbd7 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/127c0f72286541159eac49c8f958f816/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.525098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b13965-c7f1-4d0b-8805-f050d085bbd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=127c0f72286541159eac49c8f958f816, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:40.527817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b13965-c7f1-4d0b-8805-f050d085bbd7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:40.541539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 310/625] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:40 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/127c0f72286541159eac49c8f958f816/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:40.542817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32fe3b-c717-4372-91f6-a1314ef88653 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.543103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32fe3b-c717-4372-91f6-a1314ef88653 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.543375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32fe3b-c717-4372-91f6-a1314ef88653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.564678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fa32fe3b-c717-4372-91f6-a1314ef88653 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:40.565976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fa32fe3b-c717-4372-91f6-a1314ef88653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:40.609441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820b8ca-d98b-4405-8227-5f56868f722d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.644212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 311/626] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:40.645792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018bc7c-0e08-4d48-bd28-f625d7459595 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.646078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018bc7c-0e08-4d48-bd28-f625d7459595 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.646353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018bc7c-0e08-4d48-bd28-f625d7459595 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.652536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 312/627] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:40.653773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.654064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.654341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.658755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.659461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.659712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.660009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.678429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:40.681391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:40.685489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:40.686806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:40.699938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:40.704061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:40.704947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:40.705815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:40.705815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:40.705815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:40.705815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:40.708675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 316/628] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:40.709766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.763160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:40.774397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 313/629] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:40.775541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.848590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bf592-9ea0-40a9-b7c1-11188e98407b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.884541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.885197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.885478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.885981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.892277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aaade-005b-415a-aa3a-7f983a9324b2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:40.898560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:40.904246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:40.905390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 317/630] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:40.906626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.934319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:40.935320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:40.935743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:40.936184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:40.937143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:40.937605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:40.939133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:40.957066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 314/631] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:40.958205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:40.968714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.034311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933fd7d-dbc1-4e21-abae-cfb530f1bcd8 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.037209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.070522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565d852-b185-44fc-83d2-534e27b7d938 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.071056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.071707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.072014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.072283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.083387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:41.089344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:41.090512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:41.091645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:41.095466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 318/632] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:41.096356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.112075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.113054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.113542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.113960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.114820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:41.115364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.117404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.127120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 315/633] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:41.128270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.200060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.203145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b74fa6-c05e-43be-b604-3f9e108d4742 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.235557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.236149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.236427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.236700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.239086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c68a-4d56-4172-9dc6-cf1e063a76c4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.244558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.245997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:41.257116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:41.258086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:41.259035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 319/634] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:41.259630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.278414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.279081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.279368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.279643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.280515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:41.280822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.282200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6ced8-9d25-4fb6-88b2-5e6d8a451e04 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.283749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.297340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 316/635] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:41.297340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9656d9-aceb-442a-aeb1-ee8e6cc8e86d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.297340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9656d9-aceb-442a-aeb1-ee8e6cc8e86d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.297340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9656d9-aceb-442a-aeb1-ee8e6cc8e86d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.297340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 317/636] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:41.297340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9656d9-aceb-442a-aeb1-ee8e6cc8e86d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.366233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df143e-b23f-4e0a-87b6-427e76f15b53 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.399066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.399690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.399990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.400282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.403064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9656d9-aceb-442a-aeb1-ee8e6cc8e86d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.410517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.412053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:41.426667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:41.428135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:41.429227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:41.429227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:41.429227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:41.429227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:41.432821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 320/637] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:41.433632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.435307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.442149 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.442956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.443297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.443634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.444608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:41.445002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.446607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684fce-1e41-4098-851f-cae5fab94b23 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.469819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 318/638] 10.222.0.30 () {56 vars in 1137 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:41.469819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.469819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.469819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.485441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:41.486812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:41.512283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.537045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9a63-7d78-404e-b39a-437db0914c84 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.557917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 319/639] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:41.558744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.581567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.582582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.583001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.583412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.594832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.596503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:41.613679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:41.614724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:41.615730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:41.615730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:41.615730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:41.615730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:41.615730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 321/640] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:41.620413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.676884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837df1f-6af3-4bf6-a35a-30725d05d091 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.696392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.716000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.716814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.717191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.717567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.718363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:41.718769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.720324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.728471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359ab46-8e05-4321-9666-af0f1f42607c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.737319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 320/641] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:41.737319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.759110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.768001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.768626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.768904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.769175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.839723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:41.841269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:41.842104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b249f18-2205-43ba-8e02-c7e2604f56c0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:41.876126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:41.876761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:41.877040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:41.877309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:41.887573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:41.893105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:41.894278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 321/642] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:41.895121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.935046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:41.948594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 322/643] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:41.953725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:41.983140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.002532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba4d5e-e3df-4466-82e7-2b2481f56225 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.047376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.048043 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.048340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.048608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.049308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:42.049620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.051009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.058327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b7fd-ee95-47f0-8527-0745380c60a5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.067967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 322/644] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:42.067967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.098995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.099646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.099954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.100214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.107466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.108966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:42.120816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:42.121761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 323/645] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:00:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:42.122895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.175955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.180621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02524b83-5cd2-455e-9cf6-09c26e648586 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.214692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.216816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.217753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.218399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.218675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.219682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:42.220022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.222002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.226578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ffec0-52cb-46f7-a075-8ad6d8b7ac52 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.234979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 323/646] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:42.234979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.262399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.263018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.263297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.263566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.274078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.275541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:42.290528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:42.291459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:42.293130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:42.293130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:42.293130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:42.293130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:42.293130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 324/647] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:42.293130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.340649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbac9f-d9ff-421e-b881-eba40a121f47 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.374442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.375090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.375375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.375648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.383235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.384721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:42.396350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:42.397298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:42.398209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 324/648] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:42.398835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.408554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a83af5-2186-48fa-8f1d-3e8683f9927a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.424171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.443991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.444457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.444739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.445010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.445801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:42.446272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.447720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.457939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.467203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 325/649] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:42.467974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.512672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562553d-78c0-4f47-9525-41999c17d533 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.548864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.549480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.549774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.550055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.561133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.562592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.574015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ad06-5891-434f-9175-a8cb88831c67 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:42.576950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:42.578031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:42.578937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:42.578937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:42.578937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:42.578937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:42.578937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 325/650] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:42.579295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.614660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.615295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.615578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.615873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.616571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:42.616872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.618247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.629348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 326/651] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:42.630125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.651422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.683262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b522db-b407-4ae6-b9ea-d9b9c68cb217 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.686233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.716657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.717266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.717552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.717860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.718657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:42.718961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.720460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.729750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772444b-1075-44b7-8c4b-f9c56f853196 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.737336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 326/652] 10.222.0.30 () {56 vars in 1137 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:42.738139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.766711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.767268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.767553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.767858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.833593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418debb9-4170-4848-915d-b03e7bf71101 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:42.834309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.835783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.864132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.870298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:42.870907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.871181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.871451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.872211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:42.872520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:42.873883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8cef7-01b7-47e9-9724-e644a2be860f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:42.884821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 327/653] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:42.885962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889399fe-bf2a-428d-89af-909e7024e596 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:42.886245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889399fe-bf2a-428d-89af-909e7024e596 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:42.886515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889399fe-bf2a-428d-89af-909e7024e596 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:42.887959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 328/654] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:42.888516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889399fe-bf2a-428d-89af-909e7024e596 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.900771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:42.947310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 327/655] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:42.948198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:42.992500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889399fe-bf2a-428d-89af-909e7024e596 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.029933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.030510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.030791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.031065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.037781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.039240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.046570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd54b5b-e3be-4b3d-9de9-13e72323ee92 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:43.050268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:43.051222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:43.052551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:43.055940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 329/656] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:43.056995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.081409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.082044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.082333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.082607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.105526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.109871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.152039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.153574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.155016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5279e07-85f4-4dc8-ae14-cebf367cf4b5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.189335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.189995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.190262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.190534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.200067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.201572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4dde4c23-7d7b-4677-b834-68c63ea9585c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:43.213367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:43.219888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 330/657] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.220868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.221589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.234639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:43.235859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:43.252183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 328/658] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:43.252854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.309737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.311573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 331/659] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:43.312277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.354343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.359023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7333aa-24fc-4dff-af98-c7a794063515 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.397079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.397724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.398007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.398282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.408419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.409875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.412959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5bd61-757a-47be-be88-fb8b7daf7ac4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:43.424589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:43.425742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:43.426706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:43.426706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:43.426706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:43.426706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:43.427913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 329/660] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:43.429240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.449470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.450518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.450803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.451078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.452010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:43.452316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.453756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.464860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 332/661] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:43.465620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.529029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ed90-43ee-4e5f-b826-d5e2cabb90ed None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.539513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.561714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.564137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.564765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.564870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb4f15-10c3-4029-956b-ed700a3c5d03 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.565027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.565318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.566133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:43.566422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.567875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.585655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 330/662] 10.222.0.30 () {56 vars in 1139 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:43.586531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.604413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.605060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.605338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.605608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.616215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.617659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:43.631473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:43.632590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:43.633554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:43.633554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:43.633554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:43.633554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:43.633554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 333/663] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:43.633554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.686403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbea99-da2c-43b1-99e0-f8bd7ec7107c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.719083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.719733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.720041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.720311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.730400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:43.732040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f59ef3-784b-4431-9f81-8e75e6f96fc8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:43.735363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:43.736537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:43.737597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:43.739909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 331/664] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:43.741224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.766984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.767604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.767909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.768187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.768861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:43.769171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.770053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.770646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.782307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 334/665] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:43.783088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.797125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.840910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81868-047a-48cb-8ebe-4bee8e626b66 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.876168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.876796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.877076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.877345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.878119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=a88bb908a0104cc589b19095091a8a53) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:43.878425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:43.879891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:43.881196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0c4c-b410-45d7-9cec-b0d26e5d034d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:43.885995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 332/666] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/role_assignments?user.id=a88bb908a0104cc589b19095091a8a53&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:43.886754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:43.917958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:43.918597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:43.918934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:43.919211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:43.991050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:43.999923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.002050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.004444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93271bc7-8980-44dc-967b-4e48ce2612db None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.018682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.044953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.046575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.046575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.046575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.046814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.047106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.048542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.062623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 333/667] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.062623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.116328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 335/668] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:44.117109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.179589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4a1ad-f311-4645-b796-7cd6a768613e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.211941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.212555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.212833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.213103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.213812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.214114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.215504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.223591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 334/669] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.224407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.232582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc02df-865e-4d02-b735-1f823e73c022 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.240786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.248817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.272323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.272964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.273326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.273613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.280875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.282322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:44.294136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:44.295034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:44.295948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 336/670] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:44.296455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.335055 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b85b-6b78-4fff-a22f-4ed6a73a0e71 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.372277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.372845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.373125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.373397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.381243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.382682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:44.394270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:44.395189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:44.396127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 335/671] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:44.396612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.408289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58653d-c017-494a-9cc4-869b750f2e11 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.443869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.444522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.444807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.445073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.455827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.457270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:44.473352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:44.474359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:44.475285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:44.475285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:44.475285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:44.475285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:44.475285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 337/672] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:44.476031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.480134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.505894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.507270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce67fda-eac7-43ff-883f-354a1b32a81d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.541016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.541663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.541943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.542214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.553601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.555373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:44.569332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:44.570288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:44.571172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:44.571172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:44.571172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:44.571172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:44.571172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 336/673] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:44.571484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.591059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1872a7-0b0c-4027-b39d-c0240c909df6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.626677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.627223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.627505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.627787 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.628553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.628863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.630306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.644399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 338/674] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.645556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.680845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52a81a-75fa-4258-9312-28ff1cf549f5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.709004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.720061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.721092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.721381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.721659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.722778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.723096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.725279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.729257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.740287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 337/675] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.741095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.751322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eacad-8602-4263-b0f3-674dc330b070 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.786489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.787114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.787391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.787661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.788375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.788678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.790052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.802531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 339/676] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.803563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.846468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f326880-bd04-455d-a968-9f1161135be3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.881618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.882172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.882456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.882732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.883422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.883733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.885200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.900827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 338/677] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.900827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.921236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef740447-11b9-440f-a8a0-ec12bede6f60 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:44.945381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:44.966222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:44.967324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:44.967625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:44.967937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:44.968737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=5134c03bb8854cd4bee58644e6eacc51) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:44.969053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:44.971053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:44.983795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 340/678] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/role_assignments?user.id=5134c03bb8854cd4bee58644e6eacc51&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:44.983795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:44.999134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:45.028394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33a01-76ab-4224-9a79-cc84e3d04154 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.069445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.070297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.070636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.070978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.071873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=ea0805461cd3492fa5e2ebbea407e351) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:45.072258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.073911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.081507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 339/679] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/role_assignments?user.id=ea0805461cd3492fa5e2ebbea407e351&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:45.082679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:45.100228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc19eaf-c2a9-4eb2-b035-3935cbcc22e3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.138726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.139693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.140159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.140588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.157521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.159277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:45.180993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:45.181949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:45.183114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:45.185508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 341/680] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:45.192881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:45.207506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8cb89-42f8-4b47-b279-6c867c7f89be None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.222472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:45.252719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.253363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.253641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.253920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.262537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.264159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:45.276878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:45.277897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:45.278976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 340/681] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:45.289619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:45.291356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:45.328793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515fa1-9728-4bd2-b1e8-8aef5af0816d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.368106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.368730 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.369007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.369275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.370023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:45.370329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.371803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.384693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 342/682] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:45.392998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:45.399468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7165a-a4b4-42bc-855a-cf2305996c8a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.435301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.436086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.436433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.436774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.437664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:45.438041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.439560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.460459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 341/683] 10.222.0.30 () {56 vars in 1135 bytes} [Fri Aug 5 05:00:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:45.465117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:45.496845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:45.508436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881cf4-0659-46a1-9026-6847b565948b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.541695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:45.550012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.550591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.550876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.551147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.552388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=cfb671f152cd4c40825a81eec15af784) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:45.552709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.554140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.560895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 343/684] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:45 2022] GET /identity/v3/role_assignments?user.id=cfb671f152cd4c40825a81eec15af784&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:45.595332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079c680-7bf1-4bbf-b612-b44926c5034e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.635165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.636264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.636764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.637224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.714446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.716021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:45.745458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:45.821991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 342/685] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:45.827990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:45.935669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dead0c-22a7-4c52-a9c8-f0c5e756a7d8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:45.973912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:45.974815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:45.975241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:45.975666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:45.988238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:45.990015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7a4e3f35-5bcf-46b6-9f48-61e7023350bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:46.013176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:46.014121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:46.015185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:46.015185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:46.015185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:46.015185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:46.015185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 344/686] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:46.020898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:46.037583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad26c-691c-4f1d-ad27-bd0a5fd22820 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.037908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad26c-691c-4f1d-ad27-bd0a5fd22820 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.038140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad26c-691c-4f1d-ad27-bd0a5fd22820 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.039751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 345/687] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:46.044704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.044993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.045269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.068414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:46.069953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:46.126699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:46.159930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 346/688] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:46.184270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:46.212304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec2d1e-e91e-47dd-a725-67f431da3520 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:46.264417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:46.280638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:46.281626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.282121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.282606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.283497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:46.284052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:46.285633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.300420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 343/689] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:46.306802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:46.366089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5a9ba-f09f-4524-9697-e548061f38ba None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:46.421762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:46.422417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.422699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.422972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.438999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:46.440574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:46.456315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:46.457247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:46.458116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:46.458116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:46.458116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:46.458116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:46.458116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 347/690] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:46.464078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:46.497897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:46.533730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3889dd0-efde-4541-9ac1-a5e594cf579f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:46.578127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d35540-cfd4-4e29-8451-abd6ffd6ff44 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:46.614967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:46.615624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.616021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.616298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.617103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:46.617423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:46.618861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.619387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:46.620321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.620816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.621296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.632216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 348/691] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:46.637912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:46.638167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:46.639955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:46.652081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:46.652997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:46.654036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:46.656502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 344/692] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:46.662256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:46.731041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:46.748408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a953a-361a-44e3-b18d-2001f697fe5e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:46.763879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:46.786713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:46.787267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.787548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.787845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.788636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:46.790158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.824249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 349/693] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:00:46 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:00:46.854617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f0697-9416-44f7-86ee-fe1fb5ea8d9c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:46.892549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:46.893198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:46.893465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:46.893793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:46.894638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:46.894953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:46.896570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b5010-c62a-4627-a4a9-4e2aae61e0a8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:46.912026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 345/694] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:46.920982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:47.035307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c3d43-c14c-4939-bf84-826190443167 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:47.073839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:47.074394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:47.074715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:47.075110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:47.075958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=127c0f72286541159eac49c8f958f816) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:47.076271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:47.077723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91e411-868e-4d52-aa0e-891cc8d1b8ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:47.084946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 350/695] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:46 2022] GET /identity/v3/role_assignments?user.id=127c0f72286541159eac49c8f958f816&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:47.533457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60ebeb-01df-40bd-ad17-9dd1e21edc13 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:47.533738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60ebeb-01df-40bd-ad17-9dd1e21edc13 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:47.534290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60ebeb-01df-40bd-ad17-9dd1e21edc13 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:47.536064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 346/696] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:47.541423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f3304-d71e-4252-ad9d-6da2d0157413 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:47.541716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f3304-d71e-4252-ad9d-6da2d0157413 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:47.542003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f3304-d71e-4252-ad9d-6da2d0157413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:47.555687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:47.563639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-741f3304-d71e-4252-ad9d-6da2d0157413 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:47.566200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-741f3304-d71e-4252-ad9d-6da2d0157413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:47.652906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 351/697] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:47.701362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60ebeb-01df-40bd-ad17-9dd1e21edc13 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:47.708349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ff0e5-55d2-442c-ac87-f119f49631e6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:47.708889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ff0e5-55d2-442c-ac87-f119f49631e6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:47.709268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ff0e5-55d2-442c-ac87-f119f49631e6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:47.711118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 352/698] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:47.716504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:47.716792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:47.717073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:47.743086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:47.744325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:47.872403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 353/699] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:47.873489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60ebeb-01df-40bd-ad17-9dd1e21edc13 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:47.889277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:47.909277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:47.921107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:47.921780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:47.922057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:47.922327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:47.942492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:47.949646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:47.950681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:47.955105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 347/700] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:47.959881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.024130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:48.080470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f95de-814e-40bd-aed9-0991f817d13c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.122832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.123394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.123674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.124049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.124610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.126135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:48.137830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:48.138863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:48.139758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 354/701] 10.222.0.30 () {56 vars in 1129 bytes} [Fri Aug 5 05:00:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:48.144954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71dbd-f474-4e42-bd8f-1fc963b362dc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.146042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.184092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.186004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.186651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.188092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.189040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:48.189825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.192223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.209997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 348/702] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:48.215886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.265419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:48.349356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef1b7-d912-4e77-a918-76ed184e7112 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.366500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d362-bfc3-4525-b0e2-04ce5b155e1c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.384956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:48.392378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.393116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.393458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.393794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.394535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:48.394902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.396413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02227a3f-6388-4e43-a101-3f41629ee2f9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.406052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 355/703] 10.222.0.30 () {56 vars in 1124 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:48.406052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27f5e-bc03-444a-9caa-102fffb2564e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.406052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27f5e-bc03-444a-9caa-102fffb2564e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.406052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27f5e-bc03-444a-9caa-102fffb2564e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.406052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 356/704] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:48.406836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb27f5e-bc03-444a-9caa-102fffb2564e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.410867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.411651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.411973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.412250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.423701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.425434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0cf9509d-b32e-4102-8169-b6bc1b27ba83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:48.443456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:48.444454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:48.445330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:48.445330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:48.445330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:48.445330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:48.447482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 349/705] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:48.448941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.449415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.449878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.468792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:48.470253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:48.525289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb27f5e-bc03-444a-9caa-102fffb2564e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.559795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 350/706] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:48.559795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.573006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:48.574615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.575601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.576132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.576591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.577532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:48.578035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.579720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.595133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 357/707] 10.222.0.30 () {56 vars in 1124 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:48.611457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.685324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4439-1ede-4d2e-9c1b-a62290c3e92f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.727599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.728569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.728861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.729140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.729864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:48.730156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.731884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.746147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 351/708] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:48.751585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.762811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:48.868793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bec06-43ae-47c0-ab11-dcd6a4a6a581 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.874365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82824e-d568-4d40-9434-f28a4bd8e660 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:48.884272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:48.909927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.910633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.910916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.911186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.911908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.913221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.920410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:48.921394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:48.921870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:48.922275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:48.923246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:48.924127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 358/709] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/services/network => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:48.925642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:48.929812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:48.935313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:48.937108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:48.951069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:48.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:48.953054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:48.955811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 352/710] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:48.962078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.087641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8d084-1aaf-402e-8f53-c17036252f61 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.120181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:49.128391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.129298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.129796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.130275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.131925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:49.134424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:49.134424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:49.154044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 353/711] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:49.160065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.189855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4365c4-583d-4148-b0e0-7e9043a51c79 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.228587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:49.255963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.256602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.256882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.257153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.257931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] RBAC: Adding query filter params (name=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:49.258237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:49.259737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:49.272509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 359/712] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:00:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:49.277636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.284020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26c4ba-2246-4aae-aebd-ee8c51b8fe9a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.344199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.346209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.346486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.346753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.418326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:49.432361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:49.434088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:49.446141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787608-35b2-4e6d-abb8-376e74523a7c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.489987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.491251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.492122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.492606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.493505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] RBAC: Adding query filter params (type=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:49.494007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:49.496312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b7501-09fe-49e6-b77b-e13eeb2f3485 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:49.503165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 360/713] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:49.504475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631af472-f6da-41ae-aea8-72d3da9a373b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.504760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631af472-f6da-41ae-aea8-72d3da9a373b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.505034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631af472-f6da-41ae-aea8-72d3da9a373b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.507836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 361/714] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:49.512266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dfa13-4b17-4f77-9784-b56dd4b68b01 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.512857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dfa13-4b17-4f77-9784-b56dd4b68b01 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.513266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dfa13-4b17-4f77-9784-b56dd4b68b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.534579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ae1dfa13-4b17-4f77-9784-b56dd4b68b01 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:49.536137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ae1dfa13-4b17-4f77-9784-b56dd4b68b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:49.562494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 354/715] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:49.568960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.602642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:49.635791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 362/716] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:49.635791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004ef0-f214-4828-9661-7e1dbbfb88d9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.635791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004ef0-f214-4828-9661-7e1dbbfb88d9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.635791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004ef0-f214-4828-9661-7e1dbbfb88d9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.635791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 363/717] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:49.636927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.637218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.637592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.658312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:49.661754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:49.704189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:49.715584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aba3e2-bcbd-485a-b3e8-4e5ad6bb07eb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.752241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 364/718] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 05:00:49.756152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.773668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.774685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.775192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.775657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.789805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:49.791563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:49.812494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:49.813418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:49.814305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:49.814305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:49.814305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:49.814305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:49.817066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 355/719] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:49.818065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.888751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95914e70-92e9-415d-847b-67fba7a11418 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.901645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:49.928807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.929894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.930389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.930856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.932426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3298a84-2402-4a07-b978-615fbec54209 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:49.942334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:49.947870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:49.948770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:49.949623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:49.957297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 365/720] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:49.958527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.967555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:49.968243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:49.968526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:49.968802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:49.979701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:49.985244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:49.986213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:49.987086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:49.991321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 356/721] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:49.992225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:49.995676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.077212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4634d9-d14f-410d-986d-fd272a3dc9e0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.098403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0202482-e6bb-4bae-addd-c75b1fc90cdd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.113953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.114669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.115023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.115429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.116328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:50.116697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.118137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.131852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 366/722] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:50.132714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.134804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.135435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.135716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.136027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.137056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:50.137322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.138744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.147717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 357/723] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:50.148579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.175042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.216772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.254951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab6d26-9ab5-4869-9be3-ce3816048bed None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.255191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39429f86-0b15-48e1-b9f4-368c4a183740 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.288871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.289561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.289807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.290083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.293622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.294570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.294991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.295404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.296282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:50.296729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.297001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.298413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.298550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.308326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 367/724] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:50.309277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:50.310453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:50.311512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:50.312737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:50.316532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 358/725] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:50.317372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.410527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.417688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5d628-65b8-4b48-a4be-df41389c7d9c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.421660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15f63e-4dbb-429a-9696-bbea71c2f78f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.441752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.453460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.454112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.454395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.454668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.461497 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.462370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.462881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.463332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.476303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.478116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:50.495231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:50.496304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:50.497243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:50.497243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:50.497243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:50.497243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:50.503847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 368/726] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:50.505112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.588402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.589350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:50.604657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:50.605808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:50.606940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:50.606940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:50.606940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:50.606940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:50.606940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 359/727] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:50.607230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.624602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b27e55-5192-49ba-8a0a-8042bf461fe0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.663996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.665064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.665508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.665926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.666821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:50.667278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.669062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.684507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 369/728] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:50.685894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.689736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.716381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c421f99-3d16-45a6-a6fe-81922f52c42a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.752326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.752891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.753173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.753449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.754320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:50.754633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.756092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.774406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 360/729] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:50.774406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.813596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:50.883097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd223-a84a-4aa8-832c-c6499907c4ca None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:50.917549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:50.918179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:50.918461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:50.918771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:50.919813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=5134c03bb8854cd4bee58644e6eacc51) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:50.920131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:50.921709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:50.935819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 361/730] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/role_assignments?user.id=5134c03bb8854cd4bee58644e6eacc51&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:50.935819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:50.952526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d9f88-9da8-4fa6-9987-908ce6adb916 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.018846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.019416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.019698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.019999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.020708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:51.021002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.022398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06f1f6-4379-4218-814b-e6672e1139da None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.034786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 370/731] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:51.036126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd653-7bb3-457a-91f7-a2bfc8143b39 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.036418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd653-7bb3-457a-91f7-a2bfc8143b39 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.036695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd653-7bb3-457a-91f7-a2bfc8143b39 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.039807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 371/732] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:51.042218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd653-7bb3-457a-91f7-a2bfc8143b39 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.083432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.087221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149ba3-e08c-4427-926a-f9eebf57f50f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.117155 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.127713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.128666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.129090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.129504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.138500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.140523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2badc59b-c9b4-4f88-a2d5-9b648686654c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:51.156723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:51.157912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:00:51.158962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:51.163334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 362/733] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:00:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:51.165242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd653-7bb3-457a-91f7-a2bfc8143b39 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.165376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.165376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.165376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.183411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:51.184669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:51.204438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.205084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.205361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.205629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.217150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.218944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:51.235020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:51.236115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:51.237038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 372/734] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:51.237736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.263593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 363/735] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:51.265302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.357472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815ee66-8a54-4ad9-a056-b3407b051203 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.373922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.382009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.385548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59784521-1ab0-42ee-975b-5781766fa38b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.393482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.394302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.394720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.395127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.396131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:51.396574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.398242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.416747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 373/736] 10.222.0.30 () {56 vars in 1149 bytes} [Fri Aug 5 05:00:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:51.417888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.426848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.427848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.428133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.428403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.429202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:51.429505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.430941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.449329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 364/737] 10.222.0.30 () {56 vars in 1135 bytes} [Fri Aug 5 05:00:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:51.450262 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.520752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41579e9-8143-4932-b187-733d4fcd94a9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.555014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74773-392b-48d5-a16b-8727cf29e844 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.555914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.556786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.557130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.557484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.558242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.559808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.578790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 374/738] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:00:51 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:51.579721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.605673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0b91a-4e6a-434b-bb3d-616dd694b36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.607124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0b91a-4e6a-434b-bb3d-616dd694b36e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.607558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0b91a-4e6a-434b-bb3d-616dd694b36e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.608224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0b91a-4e6a-434b-bb3d-616dd694b36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.630023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.634970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.715550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0b91a-4e6a-434b-bb3d-616dd694b36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.715869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e0647-4a0a-4a37-8146-ce3a5e69a4de None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:51.717511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0b91a-4e6a-434b-bb3d-616dd694b36e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.755239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c47d58-3bb0-426f-a17e-b641641d5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:51.755926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c47d58-3bb0-426f-a17e-b641641d5c42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.756203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c47d58-3bb0-426f-a17e-b641641d5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.756475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c47d58-3bb0-426f-a17e-b641641d5c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.835485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c47d58-3bb0-426f-a17e-b641641d5c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:51.836977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c47d58-3bb0-426f-a17e-b641641d5c42 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:51.846193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 365/739] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:51 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:51.852084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd59d3-99dd-4cd4-8ba9-ec2f92182598 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.852635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd59d3-99dd-4cd4-8ba9-ec2f92182598 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.853251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd59d3-99dd-4cd4-8ba9-ec2f92182598 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.855188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 366/740] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:51.856830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd59d3-99dd-4cd4-8ba9-ec2f92182598 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:51.878987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.908291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:51.937326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 375/741] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:00:51 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 05:00:51.938611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:51.938895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:51.939163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:51.961566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:51.962757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:51.968101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd59d3-99dd-4cd4-8ba9-ec2f92182598 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.003919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.004852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.005350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.005813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.017027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.018753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:52.034089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:52.034980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:52.035880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:52.035880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:52.035880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:52.035880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:52.036120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 376/742] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 05:00:52.036120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.042451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 367/743] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:52.043710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.115207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.144966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bd19-3c63-4b19-804e-221d15535ec5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.156175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.160018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8debf-4d50-44e1-823c-756aa2e76c15 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.181525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.182176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.182490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.182769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.193358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.194829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.197545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.198420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.198906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.199385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.200335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:52.200845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.202475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:52.209994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:52.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:52.212325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:52.212325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:52.212325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:52.212325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:52.212325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 377/744] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:52.212325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.218161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 368/745] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:52.219311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.312545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca814b-d8df-4b06-8062-8f2798519cb4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.323282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988a2f7-731f-4736-a5eb-ed3ab2f95845 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.335355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.346431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.347056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.347334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.347607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.348332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.349697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.367859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 378/746] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:00:52 2022] POST /identity/v3/services => generated 240 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:52.368691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.379020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.379747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.380053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.380381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.381109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:52.381417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.382856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.397010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 369/747] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:52.403350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.420177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.485459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf5d90-bb79-4724-942c-5789f262d398 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.517443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08891a0b-d3e4-4ef3-a7a8-22bce7191487 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.551740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.553108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.553629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.554036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.554311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.554804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.555155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.559791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.567389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.569299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:52.582121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:52.583097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:52.584042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 370/748] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:52.587796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.587796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.591358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:52.602675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:52.604799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:52.607021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 379/749] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:52.611822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.670678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.691073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.727334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812b730-386d-4727-b1bd-fe6c121024d9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.743549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ca939-1fde-454e-9b2d-c706fcbb5fe1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.776074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.776972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.777265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.777539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.778331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:52.778645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.780141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.793280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.794438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.794914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.795399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.796448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:52.796964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.797788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 371/750] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:52.798853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.804028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.814187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 380/751] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:52.820377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:52.906776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e156-c160-4a5e-a766-340f0f725ff7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:52.944661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:52.951975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:52.952495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:52.952781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:52.953086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:52.953862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=127c0f72286541159eac49c8f958f816) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:52.954179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:52.955662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65c137-1d65-455f-b29e-11b669e97a0e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:52.969877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 372/752] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/role_assignments?user.id=127c0f72286541159eac49c8f958f816&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:53.015207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745abe7-941c-4fb2-87de-77d63b93b4b5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:53.023792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:53.063037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:53.063673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:53.064002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:53.064277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:53.065076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:53.065389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:53.066844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d59c54-8bfe-403e-9c73-711650a1aab7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:53.073417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 381/753] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:00:52 2022] GET /identity/v3/role_assignments?user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:53.190093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814792db-c785-49ed-88c7-6ef752840b81 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:53.190444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814792db-c785-49ed-88c7-6ef752840b81 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:53.190648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814792db-c785-49ed-88c7-6ef752840b81 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:53.192445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 373/754] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:53.198043 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:53.198326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:53.198535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:53.221719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:53.223032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:53.311320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 382/755] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:00:53.332982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814792db-c785-49ed-88c7-6ef752840b81 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:53.476451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814792db-c785-49ed-88c7-6ef752840b81 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:53.522363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:53.523038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:53.523308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:53.523578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:53.537534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:53.542348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:53.543230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:53.544765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 374/756] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:53.549422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:53.660896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:53.665160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb333eb-b34d-4cee-8c0d-0ddc54e4cebb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:53.715456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:53.716181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:53.716463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:53.716782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:53.717540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:53.717853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:53.719227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:53.728454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 383/757] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:53.736877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:53.805697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:53.853922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982af79c-f1fb-4097-bbe7-4b135d2ff240 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:53.897007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:53.897659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:53.897937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:53.898207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:53.911301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:53.912790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-99b25716-27b4-41ba-9267-937d2efa8f12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:53.932019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:53.932947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:53.934223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:53.934223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:53.934223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:53.934223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:53.934223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 375/758] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:53.940223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.007564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd955d11-3908-4a67-b980-7b17aa47467e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.007876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd955d11-3908-4a67-b980-7b17aa47467e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.008154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd955d11-3908-4a67-b980-7b17aa47467e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.009675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 376/759] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:00:54.025863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.026221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.026422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.063196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:54.068142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:54.089643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9028-980d-4fd7-966b-47c8c640344b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:54.129967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:54.131476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.132243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.132880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.134970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:54.136136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:54.138461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:54.166799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 384/760] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:54.172235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.212517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:54.216388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 377/761] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:00:54.250063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.295009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:54.358689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c855541-929e-45d8-933a-3c30bccb7b82 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:54.379841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49388-9797-4f90-96aa-9c2bbc5e01a7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:54.399783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:54.400450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.400730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.401002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.401643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:54.403029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:54.430758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:54.431793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:54.432633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 378/762] 10.222.0.30 () {56 vars in 1129 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:54.438798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.458599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:54.459255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.459533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.459833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.469361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:54.470836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:54.494711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:54.497063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:54.499293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 385/763] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:54.504803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.644166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:54.646519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6652dda-5be4-46cb-84e7-2b47a00e9e23 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:54.648939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:54.683300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bca88-f03d-40d5-860d-9c63390dd06c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:54.708690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:54.709317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.709596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.709864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.710549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:54.710852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:54.712270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:54.723879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 379/764] 10.222.0.30 () {56 vars in 1124 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:54.727910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.751710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:54.752361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.752642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.752917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.753718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:54.754029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:54.755526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:54.788396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 386/765] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:54.795232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.859827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9cfa-16ad-4e81-a1f1-7b90c7837c5a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:54.889604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:54.890238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:54.890521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:54.890838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:54.895796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:54.895796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:54.895796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:54.900550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 380/766] 10.222.0.30 () {56 vars in 1124 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:54.906529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:54.956180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.018716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eacb4a-e288-4c89-871d-31c504dc1371 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.051586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e7f84-ae2b-4e2b-b5f7-591c92117f2a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.055446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.056370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.056915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.057385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.058395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] RBAC: Adding query filter params (interface=public, service_id=7a68c6e3d99f4691a905ca6c2df53bd0, region_id=RegionOne) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:55.058904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.061606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.081582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 381/767] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/endpoints?service_id=7a68c6e3d99f4691a905ca6c2df53bd0&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:55.085410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.121733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.122474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.122720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.122996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.130349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.135721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.137846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b2fdd9b3-b911-479d-8f40-0900f916b480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:55.156938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:55.158054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:55.158955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:55.158955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:55.158955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:55.158955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:55.164262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 387/768] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:55.165903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9bfb0-aa90-422c-8742-5f28cb4485d3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.166195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9bfb0-aa90-422c-8742-5f28cb4485d3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.166476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9bfb0-aa90-422c-8742-5f28cb4485d3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.172744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 388/769] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:55.173696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9bfb0-aa90-422c-8742-5f28cb4485d3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.219291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.228346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd095f-349f-43c0-ac1a-4927544c431e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.265208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69ca79-6d99-4342-8b95-128484aa87d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.265848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69ca79-6d99-4342-8b95-128484aa87d1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.266129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69ca79-6d99-4342-8b95-128484aa87d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.266404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69ca79-6d99-4342-8b95-128484aa87d1 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.267106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69ca79-6d99-4342-8b95-128484aa87d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.268857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69ca79-6d99-4342-8b95-128484aa87d1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.279928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 382/770] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:00:55.281563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a371-86f7-4f37-a179-3b8daf459a17 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.282037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a371-86f7-4f37-a179-3b8daf459a17 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.282319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87a371-86f7-4f37-a179-3b8daf459a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.303472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9bfb0-aa90-422c-8742-5f28cb4485d3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.316230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2c87a371-86f7-4f37-a179-3b8daf459a17 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:55.317459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2c87a371-86f7-4f37-a179-3b8daf459a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:55.345843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.347085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.347566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.348100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.349553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:55.350073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.352020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.379538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 389/771] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:55.385701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.428878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 383/772] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:00:55.430217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f967604-10c1-4e7d-af6d-b2c42f59d216 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.430509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f967604-10c1-4e7d-af6d-b2c42f59d216 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.430783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f967604-10c1-4e7d-af6d-b2c42f59d216 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.436047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 384/773] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:55.437961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.438256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.438535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.458523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.477121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:55.478804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:55.508769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96277e09-3609-4d11-9b8f-8c1bddb9e0e2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.543166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.543807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.544076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.544584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.551093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.556015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 385/774] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:00:55.556160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.556259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-aa317c9c-f2b5-467c-8ef6-ad96826bcc83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:55.567754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:55.568911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:55.570170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 390/775] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:55.571549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33ca55-d201-4c84-a7b7-9d1ec08c8670 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.571925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33ca55-d201-4c84-a7b7-9d1ec08c8670 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.572370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33ca55-d201-4c84-a7b7-9d1ec08c8670 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.574002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 391/776] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:55.580488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.581810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.582147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.582554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.600372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:55.601566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:55.663868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb3f6e-4670-41df-8e6d-6e7c86302d65 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.683785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 392/777] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:00:55.683785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.703053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.704279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.704775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.705198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] PATH_INFO: `/v3/services/network` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.705993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.708253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:55.717948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:55.719173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:55.720478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:55.724111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 386/778] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:55.725083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.733849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.797040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdfb6-33f4-485c-b61f-9096b02a6dfe None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.807926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.827983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d7a7-6a8e-4ed6-b626-b336af26bd71 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.835647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.836305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.836589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.836859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.837607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:55.837913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:55.839342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:55.851527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 393/779] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:55.857144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.862861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.863520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.863824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:55.864101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:55.875135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:55.880872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:55.881783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:55.882692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 387/780] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:55.883408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:55.960424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:55.964016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d95e7-4ade-4189-8094-48d0a5eb7bc9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.990856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68c6b2-7fe2-4eec-a94a-77e5c42c2179 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:55.998868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:55.999592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:55.999959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.000237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.010663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:56.016085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:56.017046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 394/781] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:56.023706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.038797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.039450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.039830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.040263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.041014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] RBAC: Adding query filter params (name=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.041325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.042695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.047635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 388/782] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:00:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.048233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.058176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.137035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f5bb-8cdc-4824-8185-6d410e375061 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.151301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0607-4421-42a7-b913-3aaecb2e9537 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.174055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.174710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.174996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.175281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/5134c03bb8854cd4bee58644e6eacc51/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.177632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=5134c03bb8854cd4bee58644e6eacc51, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.179088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.187674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.188584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.189009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.189427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.190296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.190733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.192256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.192534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 395/783] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:56 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/5134c03bb8854cd4bee58644e6eacc51/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:56.193324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.208843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 389/784] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.210050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.230889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.310113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.336477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d4ac8-221d-4faa-832f-eeee15f80842 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.361439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd271-e78a-4c7f-a88a-ab8b4dfe1bdb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.381267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.381945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.382228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.382498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.383224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.383529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.384944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.395886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 396/785] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.396731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.410878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.411525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.411830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.412118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.412794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.413099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.414492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.420247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 390/786] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.421149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.509960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.525815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773ab62-2f46-4019-9778-c952b6eadea1 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.536250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192071-2db9-40f7-90a5-2c6260328873 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.556606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.562439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f943-a53c-4241-800a-1841b768324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.563143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f943-a53c-4241-800a-1841b768324d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.563521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f943-a53c-4241-800a-1841b768324d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.563943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f943-a53c-4241-800a-1841b768324d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.576115 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463f943-a53c-4241-800a-1841b768324d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.577711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463f943-a53c-4241-800a-1841b768324d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.583871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.585273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.585729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.586146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e463f943-a53c-4241-800a-1841b768324d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:56.595045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:56.596181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:56.597328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:56.597328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:56.597328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:56.597328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:56.601347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.603093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.603459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 391/787] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:56.604520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f943-a53c-4241-800a-1841b768324d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:56.623001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:56.624163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:56.625261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:56.625261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:56.625261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:56.625261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:56.631829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 397/788] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:56.632827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.712629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f943-a53c-4241-800a-1841b768324d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.749086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.749888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.750264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.750641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.751511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] RBAC: Adding query filter params (interface=public, service_id=e4144ecf130e4f93a8e4df75031cda5f, region_id=RegionOne) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.751951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.753422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.763523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 392/789] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/endpoints?service_id=e4144ecf130e4f93a8e4df75031cda5f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.764537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.773081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e59bd-a7c6-4d70-b641-09dd5f34dbb1 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.816451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.821049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.821899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.822716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.823280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.824346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.824666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.826130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.827867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:56.847143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 398/790] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.848075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.885339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18383a7-2251-445f-8109-212cc2895d5a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.920953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.921978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.922459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.922921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.923838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:56.924389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.925986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:56.943898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 393/791] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:56.945136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:56.953098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb68e56-c969-4ec0-9039-64a15ee1e198 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:56.988771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:56.989428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:56.989903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:56.990008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:56.990604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:56.992005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.002517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 399/792] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/services => generated 772 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:57.003505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.060587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b4958-5d20-4da4-80f2-3a8e8a0b210b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.071247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:57.106922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.108522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.109681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.110208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.112879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:57.125655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.127898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:57.156186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:57.157234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:00:57.158164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:57.159200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 394/793] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:57.166223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.207778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bfc1a-937f-4df6-a062-d2423fbec40d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.262138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.263238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.263733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.264251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.272016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.273729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:00:57.285913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:57.286867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:00:57.287859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:57.290382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 400/794] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:57.297081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.329206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15b5e7-3d3a-4e33-a62f-62f3684d1b8f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.367472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.368137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.368434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.368729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.369484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:57.369794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.371333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65c7f1-11e4-4fe0-baff-52072564eb06 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.391510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 395/795] 10.222.0.30 () {56 vars in 1135 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:57.397896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301f93-6a2b-4b00-aea8-860d2161d5ab None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.397896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301f93-6a2b-4b00-aea8-860d2161d5ab None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.397896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301f93-6a2b-4b00-aea8-860d2161d5ab None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.399689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 396/796] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:57.403862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23301f93-6a2b-4b00-aea8-860d2161d5ab None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.431892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:57.444210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3c28e-790e-48cc-9f88-74fcb88d8c79 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.467976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:57.492059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.493631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.494092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.494536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.495742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:57.496697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.498688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191d002-d9fc-4b04-90e2-412a5c0339a8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.516455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23301f93-6a2b-4b00-aea8-860d2161d5ab None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.522348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 401/797] 10.222.0.30 () {56 vars in 1149 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:57.523842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.524842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.525314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.545900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:57.547018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:57.564064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.565209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.566022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.566470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.580353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.582005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:57.601437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:57.602343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:57.603229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:57.603229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:57.603229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:57.603229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:57.605911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 397/798] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:57.606878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.634976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 402/799] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:00:57.634976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.716284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:57.744187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca53746-10c0-42ad-843d-f005c1f90af8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.744383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e36c-00c2-40e6-8cfe-f9b55f042694 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.752430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:57.789140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.789772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.790046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.790315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.791010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:57.791313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.792781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.797007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.797918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.798392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.798874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.810408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 403/800] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:57.810408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.818977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.821132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:57.841585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:57.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:57.843809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:57.843809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:57.843809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:57.843809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:57.846290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 398/801] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:57.847225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:57.924517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5963-2315-41ac-8518-22afbf60ae04 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.971536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:57.972465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:57.972869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef48c5-fa7e-41a6-9cdc-77d2bd211cd9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:57.973403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:57.973829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:57.974869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:57.977333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:57.989484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:57.990667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 05:00:57.991724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:57.994864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 404/802] 10.222.0.30 () {56 vars in 1129 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:57.995860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.023256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.030429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.030867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.031145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.031417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.038742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.040060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.045193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:58.053216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:58.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:58.055531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 399/803] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:58.056070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.142590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a145c-4f38-4251-a96f-dd9ceb233ab3 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.161530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567ccc4-a22f-41c2-8db0-d92e31034b18 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.187635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.188858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.189992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.190596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.191553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:58.192255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.194441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.198187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.198955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.199321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.199686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.200511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:58.200904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.202407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.213969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 405/805] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:58.213969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.214519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 400/805] 10.222.0.30 () {56 vars in 1124 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:58.214519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.275064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.327757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381e6e-89f2-489a-90a2-4b1e2fee4dcc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.330994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.341501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74749edb-fa61-458c-9fa2-4b56edaae515 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.366765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.367545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.367862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.368418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.369364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:58.371740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.373296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.375403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.376131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.376412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.376689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.377439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:58.377747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.379177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72da23e-3eb2-42e0-b279-3fbb1d6b3066 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.384328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 401/806] 10.222.0.30 () {56 vars in 1124 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:58.385076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.398527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 406/807] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:58.398527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cd77c-365c-4f44-b6d7-45ddf11e188b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.398527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cd77c-365c-4f44-b6d7-45ddf11e188b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.398527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cd77c-365c-4f44-b6d7-45ddf11e188b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.401468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 407/808] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:58.402220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cd77c-365c-4f44-b6d7-45ddf11e188b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.488595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619b132-d139-4e75-a8c6-e23069b7d17b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.506256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.511755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cd77c-365c-4f44-b6d7-45ddf11e188b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.524346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.524988 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.525370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.525644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.534076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.535528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.546795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f850d-35bd-48eb-9613-9973b5b5f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:00:58.546930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:58.547742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:00:58.548834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:58.549493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f850d-35bd-48eb-9613-9973b5b5f0e3 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.549493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f850d-35bd-48eb-9613-9973b5b5f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.549493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f850d-35bd-48eb-9613-9973b5b5f0e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.549493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f850d-35bd-48eb-9613-9973b5b5f0e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.550729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f850d-35bd-48eb-9613-9973b5b5f0e3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.553221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 402/809] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:58.553801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.587805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 408/810] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:00:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:00:58.587805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.587805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.587805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.599907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.606375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:58.609081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:58.702103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e05b-5928-4f3b-9de4-c62068e67e73 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.713287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 409/811] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:00:58.714251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.762084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.762846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.763131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.763410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/127c0f72286541159eac49c8f958f816/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.765502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=127c0f72286541159eac49c8f958f816, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.767048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.786866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 403/812] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:58 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/127c0f72286541159eac49c8f958f816/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:00:58.787817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.836515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.891852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c6581-2cb1-499d-a61e-6ad5b48d7965 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.924577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db656538-379d-4285-ba33-293caa1a1acc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:58.932117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:58.939892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.940864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.941448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.941891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.943361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:58.943939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:58.946100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:58.963407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 410/813] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:58.967615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:58.967615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:58.967615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:58.967615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:58.970582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:58.977729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:00:58.983353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:58.984533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:00:58.985599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:58.991939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 404/814] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:58.994337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.103576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b77684-01fa-4df5-befc-54365a7d27ac None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.109301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:59.114057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b5a92-555b-42c5-8e85-bbb2e1929331 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.143471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015c0a-0d17-4ffd-b633-106a4d4d8ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:59.144363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015c0a-0d17-4ffd-b633-106a4d4d8ac1 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.144735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015c0a-0d17-4ffd-b633-106a4d4d8ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.145100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015c0a-0d17-4ffd-b633-106a4d4d8ac1 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/8f6cb9dab5f246cabb4d8f1d7f61f20e/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.163204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:59.164321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.164763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.165134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90015c0a-0d17-4ffd-b633-106a4d4d8ac1 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=8f6cb9dab5f246cabb4d8f1d7f61f20e, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:59.165346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.166211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:59.166659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:59.166993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90015c0a-0d17-4ffd-b633-106a4d4d8ac1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:59.168483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:59.182831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 405/815] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:00:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:59.183822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 411/816] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:00:58 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/8f6cb9dab5f246cabb4d8f1d7f61f20e/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:00:59.190282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.247551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735e5b-f360-46e3-a55d-707e173bff10 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.248113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735e5b-f360-46e3-a55d-707e173bff10 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.248113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735e5b-f360-46e3-a55d-707e173bff10 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.249709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 412/817] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:00:59.255302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.255614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.255879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.290001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:00:59.291273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:00:59.340222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:59.369929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be70d2-64f4-433d-9bb4-8549df55fb4d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.390513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 413/818] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:00:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:00:59.420162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:59.420941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.421287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.421644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.422865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.444829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:59.446667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:00:59.466777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:59.467722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:59.468656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:00:59.468656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:00:59.468656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:00:59.468656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:00:59.468656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 406/819] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:00:59.479900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.508402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:59.545397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92f527-7ce0-47dd-8759-d7b62f05fc71 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.585944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:59.586598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.586881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.587150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] PATH_INFO: `/v3/services/network` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.588630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:59.590234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:00:59.611072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:00:59.612159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 05:00:59.612989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:00:59.614665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 414/820] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:00:59 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:00:59.621334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.658740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:59.681675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb9449-306f-41bd-8561-2f8c08f7acaa None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.725576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34553374-5135-4c96-aeba-740fcb41ddfd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.769663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:59.770212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.770494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.771142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.771862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:59.772173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:59.773544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:59.774211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:00:59.774845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:00:59.775120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:00:59.775386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:00:59.776112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:00:59.776410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:00:59.777771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:00:59.787542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 415/821] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:00:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:59.789598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 407/822] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:00:59.797602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.798039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:00:59.923354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:59.940424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:00:59.945276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b3409-7550-4c39-bd47-59df2e77c8f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:00:59.981446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84762ef-0970-4a4d-9375-e8d0504f6277 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:00.010629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:00.014165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:00.014165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:00.014165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:00.026320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:00.028966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:00.051252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:00.052639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:00.053584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:00.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:00.060487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:00.060797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:00.061083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:00.061781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] RBAC: Adding query filter params (type=network) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:00.062091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:00.063472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:00.063874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 416/823] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:00:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:00.066327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:00.073835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 408/824] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:00:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:00.075653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:00.212470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22fed-a99c-4203-9e5d-67a8d920d08f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:00.254226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88e5e8-cd2a-4cbb-9310-db4a71bff61b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:00.294445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:00.294990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:00.295267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:00.295647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:00.296705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:00.297024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:00.297808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:00.298441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:00.298544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:00.298724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:00.298997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:00.299716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:00.301095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:00.307598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:00.326094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 417/825] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:00.327564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:00.330082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 409/826] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:00 2022] POST /identity/v3/endpoints => generated 337 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:01:00.362417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:00.465716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b15d44-dda2-4683-b378-8f475aabf86c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:00.522967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:00.523609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:00.523922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:00.524192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:00.619364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:00.620866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:00.636550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:00.728843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 418/827] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:00.734351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:00.843726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8a312-c62a-4d96-baeb-7428477b0dae None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:00.883561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:00.884549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:00.885005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:00.885450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:00.901389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:00.903190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:00.917860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:00.918853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:00.919736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:00.919736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:00.919736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:00.919736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:00.921678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 410/828] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:00.931792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.034030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54bb8a-202f-4447-a9fd-f2b851fb51ec None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.048606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.070185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.070815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.071098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.071376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.072105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:01.072415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.073802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024c8f7-52f5-414b-bc5f-3f2d8f46984a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.086331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 419/829] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:01.092122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.140287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.198383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e2a8-32f0-4b18-8f9e-e6cb17e9ac76 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.219659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511b941-4563-410b-b077-9f347034aad0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.220012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511b941-4563-410b-b077-9f347034aad0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.220296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511b941-4563-410b-b077-9f347034aad0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.223403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 420/830] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:01.227604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909e9b9-4763-4e33-ab09-138008037eb3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.227962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909e9b9-4763-4e33-ab09-138008037eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.228246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909e9b9-4763-4e33-ab09-138008037eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.233038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.233637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.233920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.234195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.242987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.244486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.246619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b909e9b9-4763-4e33-ab09-138008037eb3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:01.248028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b909e9b9-4763-4e33-ab09-138008037eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:01.257622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:01.258842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:01.259986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 411/831] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:01.267340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.335831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 421/832] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:01.335831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e713a5-a513-45a0-8eb8-2d4ce7902edd None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.335831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e713a5-a513-45a0-8eb8-2d4ce7902edd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.335831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e713a5-a513-45a0-8eb8-2d4ce7902edd None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.339980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 422/833] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:01.341577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.342005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.342421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.360877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:01.362070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:01.382002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8763b79-070c-40c2-b5c6-6cf88a7e2243 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.385029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.419939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.420566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.420845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.421115 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.421864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:01.422350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.423675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.430294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 423/834] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:01.432034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.439816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 412/835] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:01.444141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.475117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.538443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f9e38-a36f-4153-8508-ad7c1ae83624 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.544454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc354e1e-04c2-41e8-ab09-50c9abc0b2a9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.574152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.574794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.575133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.575399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.576059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.577427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:01.586050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:01.587035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:01.587907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 424/836] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:01.588694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.588780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.588849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.589268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.589625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.590849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=5134c03bb8854cd4bee58644e6eacc51) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:01.591170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.592625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e81b-6dc0-44ee-9a27-5c58153cf14f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.619805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 413/837] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/role_assignments?user.id=5134c03bb8854cd4bee58644e6eacc51&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:01.619805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264f0ed-d3c9-4fad-a375-68e76ad48801 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.619805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264f0ed-d3c9-4fad-a375-68e76ad48801 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.619805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264f0ed-d3c9-4fad-a375-68e76ad48801 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.619805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 414/838] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:01.619805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264f0ed-d3c9-4fad-a375-68e76ad48801 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.645875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.707165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.711532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ec34-1bb7-4552-b877-89549a3eccaa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.723181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264f0ed-d3c9-4fad-a375-68e76ad48801 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.751091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced17a8-8b18-4a41-a729-8961df96376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.751907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced17a8-8b18-4a41-a729-8961df96376a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.752194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced17a8-8b18-4a41-a729-8961df96376a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.752470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced17a8-8b18-4a41-a729-8961df96376a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.763243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.763913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.764193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.764461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.764629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced17a8-8b18-4a41-a729-8961df96376a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:01.765137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:01.765433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.766802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8ced17a8-8b18-4a41-a729-8961df96376a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:01.770008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:01.770893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:01.771734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 425/839] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:01.772489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 415/840] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:01.773190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.773476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.773754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.777775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.794222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:01.795474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:01.866072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 426/841] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:01.866933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.891519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:01.899542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25689a91-7548-4b21-adfe-0ffeac7d401f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:01.934108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:01.934841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:01.935118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:01.935388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:01.936130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:01.936438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:01.937805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:01.952180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 416/842] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:01.952180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:01.990689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.002608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e31a6-8ba5-4663-9f0e-f2928de7ccf2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.041564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.042191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.042467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.042740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.043440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:02.043791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.045214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.050326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 427/843] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:02.057558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.059139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c11550-6dae-45ac-9aba-8734879eb4fb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.097996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.098649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.098937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.099209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.114067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:02.121059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:02.122185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:02.123053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 417/844] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:01:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:02.128507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.151858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.192814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102b051-aeb0-4c6a-bbfc-cca66bd332f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.233976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.234624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.234906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.235179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.246908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.248254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:02.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:02.264823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:02.265720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:02.265720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:02.265720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:02.265720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:02.266388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 428/845] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:02.272105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.273407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc407f3-7057-4f5d-874e-905f328ba1b4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.285462 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.323396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.324085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.324366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.324640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.325429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:02.325739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.327238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.336491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 418/846] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:02.342131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.390414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27df71-ba89-4279-907b-5d68233c1705 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.424282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.426495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.427753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.428341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.428822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.429710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:02.430227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.431973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.445069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 429/847] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:02.451918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.452636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be6c4e-d698-4a0f-84d5-b8e310079d70 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.488339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.489646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.489646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.489646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.501127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.502614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:02.518198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:02.519287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:02.520393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:02.520393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:02.520393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:02.520393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:02.524092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 419/848] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:02.524796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.526067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.563449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b088f-1567-461a-bd18-636e46cd5a23 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.601568 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.602466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.602921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.603416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.611585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.613541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:02.626627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:02.627847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 05:01:02.629020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:02.631231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 430/849] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:02.636419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.636555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a94a46-fb74-4c96-9155-4f0ec39dc64d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.664135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.674447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.675037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.675318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.675593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.676347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:02.676669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.678487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.695648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 420/850] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:02.696575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.752803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed209f-a2a8-4e7e-b8f6-7d129948123c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.779539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:02.791314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.792254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.792688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.792966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.793807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:02.794209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.795749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.812554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fcca7-9657-4128-bed2-dedac75eb49f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.818589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 431/851] 10.222.0.30 () {56 vars in 1135 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:02.819706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.869878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.871430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.872005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.872475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.882589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:02.885062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:02.902885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:02.903896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:02.904846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:02.910424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 421/852] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:02.910494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:02.931209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440c0c0-e9c3-48e4-92f0-735c15bdec3e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:02.966274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:02.966953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:02.967234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:02.967503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:02.979913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.028122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.047877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.049554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.054084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb9a71-7e9b-4793-952f-f141627fc29c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.099889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.101534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.102361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.102816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.103813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:03.104296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.106770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.135926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 422/853] 10.222.0.30 () {56 vars in 1149 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:03.138385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.159799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 432/854] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:01:03.164902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.254064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.271492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f899dd8-c6c9-4112-b4af-26f67e103524 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.277213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7db1f-2a4d-44c4-80b7-ddac48afcccd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.280004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.309667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.310686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.311178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.311641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.324314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.325041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.325350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.325862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.339048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.340776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:03.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:03.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:03.361971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:03.361971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:03.361971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:03.361971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:03.363952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 433/855] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:03.366929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.393341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.395925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.480174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63801f-d529-4df7-994f-678b68dc0b52 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.514704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 423/856] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:03.515717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.522393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.523217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.523662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.523900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.524253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.525571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:03.526028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.528010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.530183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.549806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 434/857] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:03.550843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.649064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd88465-d3a1-4f56-8b53-a953c7a8db17 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.661287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628632e-1b3b-43c5-8701-68e573a24864 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.690860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.691509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.691804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.692083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.701162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.701803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.702077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.702338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.703032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.704524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.709186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.710602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:03.720799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:03.721722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:03.722593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:03.722593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:03.722593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:03.722593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-921105e9-d0d3-4634-848a-4b693309c854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:03.722822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:03.723626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 435/858] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c94e0-7a2d-4664-99c8-18f92d125ebb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.729842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c94e0-7a2d-4664-99c8-18f92d125ebb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.731848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c94e0-7a2d-4664-99c8-18f92d125ebb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.732007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 424/859] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:03.732158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 436/860] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:03.732301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.732657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c94e0-7a2d-4664-99c8-18f92d125ebb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.792886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.793198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:03.840631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c94e0-7a2d-4664-99c8-18f92d125ebb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.841907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957febd-cf87-4b62-b70b-8f566a7928c6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:03.877579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.878203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.878484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.878755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.879448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:03.879749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.880845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:03.881139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcefb99-e36e-4d3a-8fb2-245c2faf6eb4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.881467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.881745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.882018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.882763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:03.883067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:03.884580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:03.898325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 425/861] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:03.898325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:03.898325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:03.898325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:03.901947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 437/862] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:03.902706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:03.921996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:03.923184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:03.996135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 426/863] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:03.996336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.022502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad52f13-7a0b-498e-a627-43eb6c4b47aa None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.038065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:04.056017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:04.060482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.061143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.061430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.061701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.069045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.070569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:04.081746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:04.082671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:04.083627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 438/864] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:04.084146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.114382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64fbd-6dbc-4220-b686-c9050d0efc1f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.150644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.151271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.151553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.151854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.152646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=127c0f72286541159eac49c8f958f816) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:04.152990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.154459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.160346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 427/865] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:03 2022] GET /identity/v3/role_assignments?user.id=127c0f72286541159eac49c8f958f816&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:04.161117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.194274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d23cb-583f-4810-b34f-a1e5ef6a57a2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.233502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.234181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.234522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.234780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.235453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.237077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-787176cd-8e2d-4f30-915c-79593dee4e88 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:04.247143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:04.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:04.249109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 439/866] 10.222.0.30 () {56 vars in 1132 bytes} [Fri Aug 5 05:01:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:04.249953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27479e-d47e-432e-845a-fb4d2e9d9075 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.250604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27479e-d47e-432e-845a-fb4d2e9d9075 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.250604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27479e-d47e-432e-845a-fb4d2e9d9075 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.255806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 440/867] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:04.255806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27479e-d47e-432e-845a-fb4d2e9d9075 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.281507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee000468-2f6e-49e4-9f1e-50566d144127 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.297238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:04.300905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:04.325192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.326389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.327103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.328825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.328825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:04.328825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.329920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c9c37-5a67-461c-83ab-c490e324c43a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.344470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 428/868] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:04.345783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.346050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.346287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.364669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:04.365746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:04.383409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27479e-d47e-432e-845a-fb4d2e9d9075 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.422809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.423428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.423707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.424007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.424747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:04.425157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.426624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.437406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 441/869] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:04.437406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.456217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 429/870] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:04.457166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.552324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b70ceb-47ac-445d-90ce-3ab1c4cdc16d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.554734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:04.563314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245b03-577e-41c8-8729-6f052a53bdd4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.565388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:04.588913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.589457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.589732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.590004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.590818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:04.591128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.592584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.599010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 442/871] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:04 2022] GET /identity/v3/role_assignments?user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:04.600029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:04.610658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.611315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.611596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.611892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.612593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:04.612898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.614932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957f3a-4862-4ad2-b17d-ef6b0d7b09a7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.623965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 430/872] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:04.737900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf1cb-b5c6-4300-bc3f-8fb382d04452 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:04.783646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e51dd-aa82-4201-8926-208c71f877e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:04.784338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e51dd-aa82-4201-8926-208c71f877e0 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:04.784723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e51dd-aa82-4201-8926-208c71f877e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:04.784997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e51dd-aa82-4201-8926-208c71f877e0 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:04.785715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e51dd-aa82-4201-8926-208c71f877e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:04.787141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e51dd-aa82-4201-8926-208c71f877e0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:04.805116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 443/873] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:04 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:06.231436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949edcb-d643-4095-8b3a-7730043049e7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.232474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949edcb-d643-4095-8b3a-7730043049e7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.232970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949edcb-d643-4095-8b3a-7730043049e7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.234774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 431/874] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:06.240658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.241590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.242064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.260817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:06.262421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:06.284455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:06.339397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 444/875] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:06.361326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949edcb-d643-4095-8b3a-7730043049e7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:06.487791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949edcb-d643-4095-8b3a-7730043049e7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:06.523651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:06.524253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.524539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.524816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.525455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:06.526839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-390bb47a-fd7b-48c6-8c70-9e5b3f08ec4f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:06.537567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:06.538574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:06.539392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 432/876] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:06.546585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:06.687992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b34e4b-e8f7-4cf6-9544-50248d6a5d79 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.688349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b34e4b-e8f7-4cf6-9544-50248d6a5d79 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.688556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b34e4b-e8f7-4cf6-9544-50248d6a5d79 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.690179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 433/877] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:06.696410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239195e7-ef7b-44a7-89e8-e1eba9279f87 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:06.699172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b88c5-421c-47a1-8a9c-9009e2bc9d79 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.699459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b88c5-421c-47a1-8a9c-9009e2bc9d79 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.699739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b88c5-421c-47a1-8a9c-9009e2bc9d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.732534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a49b88c5-421c-47a1-8a9c-9009e2bc9d79 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:06.734346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a49b88c5-421c-47a1-8a9c-9009e2bc9d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:06.743545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:06.744203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.744483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.744754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.745441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:06.745742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:06.747208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859ace-961c-42a5-be85-8c5e9d079520 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:06.753199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 445/878] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:06.754518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354a16-3505-4762-b6e2-00af5ae125ac None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.754802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354a16-3505-4762-b6e2-00af5ae125ac None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.755079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354a16-3505-4762-b6e2-00af5ae125ac None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.756537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 446/879] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:06.758416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9354a16-3505-4762-b6e2-00af5ae125ac None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:06.775521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:06.777595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:06.832325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 434/880] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:06.833779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08521343-5d08-440f-b57f-6a0a88bc9a0d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.834137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08521343-5d08-440f-b57f-6a0a88bc9a0d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.834492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08521343-5d08-440f-b57f-6a0a88bc9a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.854516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-08521343-5d08-440f-b57f-6a0a88bc9a0d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:06.855724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-08521343-5d08-440f-b57f-6a0a88bc9a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:06.885434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9354a16-3505-4762-b6e2-00af5ae125ac None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:06.923192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:06.923863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.924120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.924383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.925062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:06.925365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:06.927092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:06.931678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 447/881] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:06.932455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:06.952349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 435/882] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:06.960474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb84961-9384-4f5f-aef9-78c211f9d60b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.960474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb84961-9384-4f5f-aef9-78c211f9d60b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.960474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb84961-9384-4f5f-aef9-78c211f9d60b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:06.960474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 436/883] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:06.966345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:06.966663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:06.966903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.002415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:07.002415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:07.046569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.068789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a427b18-1918-4061-a997-c1e7fdeb0ccb None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.092140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 437/884] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:07.093067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.110225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.110855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.111129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.111395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.112121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.113501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.119343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.135202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 448/885] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:06 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:07.135578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.212577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171d60d-ad28-45f5-99e1-17c05d8cb7fe None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.273555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.274641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.275103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.275928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.288600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573238a-512a-4543-ab65-fbbf11793d59 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.291589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:07.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:07.300185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:07.301228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:07.304952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 438/886] 10.222.0.30 () {56 vars in 1120 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:07.310557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.344300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.345249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.345472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.345756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.346404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.347820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:07.357130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:07.358126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:07.359123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 449/887] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:07.364192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.370219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.412453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.442131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466f24-35f2-4029-94ae-ed6cd99fc029 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.479002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d33bb-fa5a-4db5-959c-90ff2ccba7b7 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.480934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.481874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.482323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.482790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.483673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:07.484200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.485792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.494877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 439/888] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:07.500275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.514233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.515147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.515605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.516111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.516986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:07.517480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.519123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.524518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 450/889] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:07.530361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.598590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.607970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa64791-39a9-41fc-b415-e05fbd5ab373 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.633763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca1d6e-d2a5-4c40-b969-b384dacf674b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.644146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.645171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.645655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.646127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.656151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.660310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.663141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.671665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.672660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.673088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.673506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.674444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:07.674895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.676467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:07.682216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:07.683098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:07.684026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:07.684026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:07.684026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:07.684026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:07.687237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 451/890] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:07.687362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 440/891] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:07.688137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.689844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.792901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aef73-4d82-4a61-86cd-48f563d9e209 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.815386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1fc58-b379-4cd1-a698-8aa6a2b8c916 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.832115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.833508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.834295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.834656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.834940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.836427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] RBAC: Adding query filter params (interface=public, service_id=d1b5bc2e7db3423695c782778275ded6, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:07.836745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.838417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.852567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 452/892] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/endpoints?service_id=d1b5bc2e7db3423695c782778275ded6&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:07.854559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.855449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:07.856484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:07.856841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:07.857187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:07.857958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:07.858335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:07.860386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:07.873593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 441/893] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:07.880523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:07.935118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:07.986082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa85-99e0-44e6-9ed4-17589f9d10f4 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:07.990571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63d7af-366a-47ae-9c36-c7c78fb94e9b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:08.027472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:08.028421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.028837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.029283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.030083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:08.031596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:08.037932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 453/894] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:01:08.047110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:08.048119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.048536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.048948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.058128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:08.060045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:08.074208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:08.075547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:08.076789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:08.079290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 442/895] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:01:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:08.084854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:08.150230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:08.196441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d048003-5ba0-40d5-818f-ad0acf5f1455 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:08.239278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:08.239944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.240240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.240507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.241313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:08.241630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:08.243097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:08.261582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 454/896] 10.222.0.30 () {56 vars in 1149 bytes} [Fri Aug 5 05:01:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:08.267455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:08.381234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9a1a-6226-4dfd-9415-88aafa1c3347 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:08.394208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:08.424457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:08.425653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.426320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.427412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.498049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:08.499794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:08.603958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 443/897] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:08 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:08.605903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:08.713846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a2c9-804f-464e-b77a-402e5bb92864 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:08.733110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:08.751035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf32708-a396-406e-9768-9569923e1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:08.751681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf32708-a396-406e-9768-9569923e1673 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.751981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf32708-a396-406e-9768-9569923e1673 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.752254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf32708-a396-406e-9768-9569923e1673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.762491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf32708-a396-406e-9768-9569923e1673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:08.764048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf32708-a396-406e-9768-9569923e1673 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-acf32708-a396-406e-9768-9569923e1673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:08.778840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:08.779873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:08.780760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:08.780760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:08.780760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:08.780760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:08.780760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 455/898] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:08.787248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:08.799818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:08.891455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576125a-408d-4ef2-b8a5-28c97a3a9b02 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:08.927463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:08.928513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.928996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.929469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.930359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:08.930864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:08.932523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31340245-3a31-4a7b-b7d9-8c59292c2eea None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:08.945252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 444/899] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:08.951181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf32708-a396-406e-9768-9569923e1673 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:08.962845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d36da-5388-4a4b-904b-316f70db6f04 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.963140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d36da-5388-4a4b-904b-316f70db6f04 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.963419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d36da-5388-4a4b-904b-316f70db6f04 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.967904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 445/900] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:08.970285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:08.970575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:08.970856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:08.995396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:08.997714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:09.063752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf32708-a396-406e-9768-9569923e1673 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.083163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:09.087904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 446/901] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:09.106537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.107990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.108662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.109410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.110511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.117230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.119008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:09.130848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:09.131723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:09.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:09.134987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 456/902] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:09.141516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.173087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:09.224831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4ade1-4ebd-41f2-96bd-ed07efad9e39 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.245999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57eae-29ca-4f76-af58-a230b09e4f18 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.263913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a0dee-9a79-4ec7-9433-f1aff51e41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.264531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a0dee-9a79-4ec7-9433-f1aff51e41a8 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.264811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a0dee-9a79-4ec7-9433-f1aff51e41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.265250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a0dee-9a79-4ec7-9433-f1aff51e41a8 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.265769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a0dee-9a79-4ec7-9433-f1aff51e41a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.267129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a0dee-9a79-4ec7-9433-f1aff51e41a8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.287650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.290462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 447/903] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:09 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:01:09.290583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.290583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.290583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.290842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c516d2-8b2e-4c88-a4f7-117c222ab6a2 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.290842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c516d2-8b2e-4c88-a4f7-117c222ab6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.290842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c516d2-8b2e-4c88-a4f7-117c222ab6a2 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.291064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:09.291064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.291214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 448/904] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:09.291294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.292908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.292908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.293225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.312470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 457/905] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:09.313977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:09.315293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:09.318626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.381674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:09.408187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 449/906] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:09.424222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.436476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:09.449199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108379d2-4d9b-463b-adbb-5936a2465d52 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.491891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.492737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.493176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.493483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.494743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] RBAC: Adding query filter params (role.id=5493cd3a48994597ab746dd76d27c6ea, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:09.495225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.497655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.507868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 458/907] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:09 2022] GET /identity/v3/role_assignments?user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=5493cd3a48994597ab746dd76d27c6ea => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:09.539393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23e1ad-4fab-4384-8838-aad838af1b41 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.576265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.576934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.577211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.577487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.578128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.579546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:09.589404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:09.590709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:09.591807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 450/908] 10.222.0.30 () {56 vars in 1132 bytes} [Fri Aug 5 05:01:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:09.597339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.704604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b65c-e36a-4549-8093-819ec0cb2f99 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.739644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.740421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.740705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.741041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.742052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:09.742401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.744169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.751558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 459/909] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:09.757317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.813454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:09.869238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6377-68bf-4879-ac65-ac2d02795e3a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:09.914898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:09.916220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:09.916942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:09.917592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:09.918903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:09.919590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:09.921874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:09.929950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 451/910] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:09.935622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:09.959799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:10.053231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3570a6d-dd51-45cb-a498-f72fe12af782 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:10.087860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:10.088497 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.088781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.089055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.089812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] RBAC: Adding query filter params (interface=public, service_id=96398e370e5d44c5ad1b9a7afadb45a5, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:10.090122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:10.091591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c348b-29c8-46cb-9039-9f72d6d3636c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:10.098455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 460/911] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:01:09 2022] GET /identity/v3/endpoints?service_id=96398e370e5d44c5ad1b9a7afadb45a5&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:10.103788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:10.224690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41653cc-638c-4c4e-9db9-f92ff83cc0ca None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:10.251881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:10.261662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:10.262329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.262775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.263053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.263783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:10.265461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8897d-ffff-471f-ab7e-7397f5083b7a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.265744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8897d-ffff-471f-ab7e-7397f5083b7a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.266012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8897d-ffff-471f-ab7e-7397f5083b7a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.267487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 461/912] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:10.268743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:10.274082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.274375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.274657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.276177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 452/913] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:01:10 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:01:10.294035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:10.295287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:10.371597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 462/914] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:10.395230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:10.505907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20212986-0e7d-42a0-bbf5-fab87eea7ffb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:10.541830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:10.542492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.542774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.543045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.543746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:10.545142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:10.554740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:10.556059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:10.556976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 453/915] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:10.564175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:10.588288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:10.676281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0f80e-522f-4b7b-842d-57f5c7396f8d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:10.711048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:10.712535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.713021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.713493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.714394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:10.714906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:10.716614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:10.723014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 463/916] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:10.729346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:10.774874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:10.835502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c2360-0842-48cd-aa6b-260231304d00 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:10.870466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:10.871453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:10.871993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:10.872475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:10.873342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:10.873846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:10.875413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc222-e193-4aaa-b621-0ad557509bea None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:10.882673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 454/917] 10.222.0.30 () {56 vars in 1114 bytes} [Fri Aug 5 05:01:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:10.888580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:10.965353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:10.988309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaec183-1a76-4e7b-bb2e-bd3999dfb351 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:11.034761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b6b0b-b3ba-4c82-b383-efb6fb8247ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:11.035313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b6b0b-b3ba-4c82-b383-efb6fb8247ae None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.035940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b6b0b-b3ba-4c82-b383-efb6fb8247ae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.036214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b6b0b-b3ba-4c82-b383-efb6fb8247ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.036942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b6b0b-b3ba-4c82-b383-efb6fb8247ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:11.038316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b6b0b-b3ba-4c82-b383-efb6fb8247ae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:11.061672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 464/918] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:11.496060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc4253-9a82-49cb-a4f0-7bac694c8c69 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.496644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc4253-9a82-49cb-a4f0-7bac694c8c69 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.496644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc4253-9a82-49cb-a4f0-7bac694c8c69 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.498259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 455/919] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:11.503646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609897e0-2ccf-4b0c-a930-4fd62530192d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.503961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609897e0-2ccf-4b0c-a930-4fd62530192d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.504240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609897e0-2ccf-4b0c-a930-4fd62530192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.522751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-609897e0-2ccf-4b0c-a930-4fd62530192d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:11.524030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-609897e0-2ccf-4b0c-a930-4fd62530192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:11.599199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 465/920] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:11.621167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc4253-9a82-49cb-a4f0-7bac694c8c69 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:11.652881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec090c-fbb2-4575-8e14-230d478df307 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.653402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec090c-fbb2-4575-8e14-230d478df307 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.653840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec090c-fbb2-4575-8e14-230d478df307 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.655778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 466/921] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:11.661532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.661891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.662226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.681349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:11.682723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:11.740758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:11.746170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc4253-9a82-49cb-a4f0-7bac694c8c69 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:11.755110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:11.762079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 467/922] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:11.782578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:11.783233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.783515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.783739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:11.783886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.784427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:11.785800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:11.801451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:11.802847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:11.803714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 456/923] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:11.809649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:11.894569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58883a5-f16a-4d08-b37d-612695123b51 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:11.930917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:11.931152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776e0a-9243-4bc7-93c4-8f7980381454 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:11.932205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.932818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.933376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.948474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:11.954615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:11.955661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:11.956539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:11.959829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 468/924] 10.222.0.30 () {56 vars in 1120 bytes} [Fri Aug 5 05:01:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:11.961705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:11.976709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:11.977350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:11.977632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:11.977898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:11.978755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:11.979065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:11.981171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:11.987905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 457/925] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:11.992633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.023363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.043628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.091955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24efedc-875b-43e7-b6e0-77ae019a468a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.115504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546930a-925e-41bd-b7a5-5ce524145638 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.138358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.139126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.139739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.140131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.141358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:12.141763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.143741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.152219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.153160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.153650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.154122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.154992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:12.155495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.157116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.159414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 469/926] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:01:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:12.160734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.168641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 458/927] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:12.169762 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.275427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.282252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2f6e0-da41-4449-acfd-f605dbf28700 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.298723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a043de7-1f45-469f-aadc-1a29bbd8e6f9 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.314107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.320690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.321295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.321570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.321842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.322600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] RBAC: Adding query filter params (interface=public, service_id=1edb10db620e4963b958a5812abcf567, region_id=RegionOne) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:12.322910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.324511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.335383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 459/928] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/endpoints?service_id=1edb10db620e4963b958a5812abcf567&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:12.336311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.344446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.345965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.346458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.346926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.363479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.365934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:12.385084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:12.385992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:12.386886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:12.386886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:12.386886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:12.386886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:12.394340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 470/929] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:12.395081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.456706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176fdd6-d114-4f96-a0d1-ff5f40707204 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.500839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda6d39-dd5d-47b4-88ad-986e77cf9682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.502579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda6d39-dd5d-47b4-88ad-986e77cf9682 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.503056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda6d39-dd5d-47b4-88ad-986e77cf9682 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.503543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda6d39-dd5d-47b4-88ad-986e77cf9682 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.504444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda6d39-dd5d-47b4-88ad-986e77cf9682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.506590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda6d39-dd5d-47b4-88ad-986e77cf9682 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.515466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 460/930] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:01:12.525366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9704bc-eb2d-41c7-91d8-9f46310a7c38 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.572954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7247d-55fd-4aff-8344-8d0ea5f2109e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.573145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7247d-55fd-4aff-8344-8d0ea5f2109e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.573415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7247d-55fd-4aff-8344-8d0ea5f2109e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.579089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 461/931] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:12.580909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.581294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.581656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.584046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.585307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.585983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.586575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.587826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:12.588672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.591288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.602463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:12.604224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:12.612551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 471/932] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:12.620039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.629355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.655403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.687460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 462/933] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:12.711507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.749013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff47fd-a775-4547-85bd-9190b4158f60 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.783572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.784258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.784542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.784818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.792869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.793986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:12.806598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:12.807560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:12.808789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 472/934] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:12.813890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.852281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de981-68d3-4561-91cd-479f30968280 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.897200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.898572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.899011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.899830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.900894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.902874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:12.903850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:12.914957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:12.916095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 05:01:12.916929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:12.919856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 463/935] 10.222.0.30 () {56 vars in 1132 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:12.923363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:12.934221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a179c14-90b4-453a-ac8f-4ff9d618a26b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:12.954429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:12.972397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:12.973066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:12.973709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:12.973985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:12.974774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:12.975082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:12.976590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.003285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 473/936] 10.222.0.30 () {56 vars in 1149 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:13.008266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.073093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df3a9-116a-4122-8938-5b7b8764ffde None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.114349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567de81-2385-4fd3-8e7d-2a3244242008 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.119164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.120018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.120450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.120858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.121723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:13.122164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.124297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.131655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 464/937] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:13.137253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.156729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.157472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.157820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.158154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.167468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:13.169866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.171633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:13.189440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:13.190361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:13.191230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:13.191230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:13.191230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:13.191230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:13.195512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 474/938] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:13.196513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.250738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:13.276113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81542444-5591-48f1-ae62-1bdee6b6d075 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.312440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.313233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.313577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.313918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.314661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:13.315052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.316614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.326956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 465/939] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:13.328054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.331256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2df51b-2967-436a-9884-dc8407cf7515 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.382183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.382731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.383008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.383278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.384000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:13.384307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.385690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.406025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 475/940] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:13.412060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.444959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9182f2-a32e-4f85-90db-f274fffaae3d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.479223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f58bad-3f01-4839-a3cc-adc255a60fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.479884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58bad-3f01-4839-a3cc-adc255a60fd4 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.480167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58bad-3f01-4839-a3cc-adc255a60fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.480443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58bad-3f01-4839-a3cc-adc255a60fd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.481198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f58bad-3f01-4839-a3cc-adc255a60fd4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.482333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:13.482579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f58bad-3f01-4839-a3cc-adc255a60fd4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.508713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 466/941] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:13.510967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e63140-4ea5-42e8-92ee-8ffb5d8181c3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.511517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e63140-4ea5-42e8-92ee-8ffb5d8181c3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.512136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e63140-4ea5-42e8-92ee-8ffb5d8181c3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.513951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 467/942] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:13.519899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.520243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.520451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.534106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:13.547610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:13.548941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:13.566413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406971ec-0e2b-4b1c-b610-bb3a72b45fde None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.628946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.630035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.630904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.631608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.640256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 468/943] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:13.641037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.642855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:13.656094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:13.657186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:13.658117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:13.658572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.661324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 476/944] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:13.669474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.797936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdb282-0e46-4d09-9eac-24c918956d00 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.806163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ccb03-bd3d-42a7-8417-385fc8d98c5f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:13.809444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:13.843078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.844155 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.844592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.845048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.846515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:13.847171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.849145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.849730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:13.857475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:13.858440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:13.858941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:13.859513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:13.860446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:13.862215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:13.865395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING keystone.server.flask.common [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 05:01:13.869098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 477/945] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:13.880494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:13.886441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.sql.core [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 05:01:13.886441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 05:01:13.886441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [parameters: {'id': '0d9d216be72e41259479d46b41d50608', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 05:01:13.886441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80379) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d9d216be72e41259479d46b41d50608.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d9d216be72e41259479d46b41d50608. Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application conn.query(q) Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 05:01:13.905735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result.read() Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 05:01:13.906640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application next(self.gen) Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application yield resource Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application next(self.gen) Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise self.value Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application session.commit() Aug 05 05:01:13.907644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.session.flush() Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application compat.raise_( Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 05:01:13.908598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 05:01:13.909531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application conn.query(q) Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application result.read() Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 05:01:13.910353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [parameters: {'id': '0d9d216be72e41259479d46b41d50608', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:13.911252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:13.912227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d9d216be72e41259479d46b41d50608. Aug 05 05:01:13.913351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:13.915882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 469/946] 10.222.0.30 () {60 vars in 1140 bytes} [Fri Aug 5 05:01:13 2022] POST /identity/v3/projects => generated 262 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:13.924934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:14.004003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d167d46-72fc-4a16-9fea-f39ad1836763 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:14.032013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21c52-aee4-4682-96c8-9df790a7eccf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:14.038892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:14.039714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:14.040467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:14.040467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/8f6cb9dab5f246cabb4d8f1d7f61f20e/roles/5493cd3a48994597ab746dd76d27c6ea` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:14.042923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=8f6cb9dab5f246cabb4d8f1d7f61f20e, role_id=5493cd3a48994597ab746dd76d27c6ea)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:14.044602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:14.062765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 478/947] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:01:13 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/8f6cb9dab5f246cabb4d8f1d7f61f20e/roles/5493cd3a48994597ab746dd76d27c6ea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:01:14.088964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:14.089195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:14.089539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:14.089900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:14.102734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:14.104719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c9ad29ac-7a40-4f14-96e6-40e1c9b0bc56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:14.119920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:14.121170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:14.122448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:14.124867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 470/948] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:14.129744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:14.173406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:14.252420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946f4d6-9d10-4609-889f-e89566331a26 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:14.290663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:14.291413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:14.291749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:14.292109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:14.292889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:14.293282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:14.295195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ff15c-a709-4a9b-abe7-300cab4543ae None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:14.314571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 479/949] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:14.638937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b81b8-cb25-49aa-89ce-08ab5368a476 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:14.639472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b81b8-cb25-49aa-89ce-08ab5368a476 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:14.639472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b81b8-cb25-49aa-89ce-08ab5368a476 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:14.643836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 471/950] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:14.649552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:14.649843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:14.650119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:14.668974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:14.670156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:14.738844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 480/951] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:14.761046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b81b8-cb25-49aa-89ce-08ab5368a476 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:14.816190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:14.875412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b81b8-cb25-49aa-89ce-08ab5368a476 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:14.912235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:14.912904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:14.913188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:14.913463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:14.914101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:14.915479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:14.924668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:14.925842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:14.927062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 472/952] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:14.932317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:14.952288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:15.048278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59140cb-620d-43d3-935b-4b4c9c778591 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:15.082860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:15.083493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:15.083786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:15.084145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:15.084819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:15.085120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:15.086487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:15.091502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 481/953] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:15.100968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:15.207831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93b2c2-2d8a-48be-9c4d-663870157485 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:15.219746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:15.243755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:15.244477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:15.244931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:15.245217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:15.245906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:15.246213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:15.247623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c391f3-88b6-4535-895a-0cd17d0ff90b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:15.253365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 473/954] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:15.258846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:15.346632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:15.365024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa50c-e35a-4a6b-97ce-5bea0d8fe9d5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:15.400056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765320-a191-486d-9295-4cbcafce51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:15.400681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765320-a191-486d-9295-4cbcafce51d1 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:15.400961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765320-a191-486d-9295-4cbcafce51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:15.401232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765320-a191-486d-9295-4cbcafce51d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:15.402011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765320-a191-486d-9295-4cbcafce51d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:15.403487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765320-a191-486d-9295-4cbcafce51d1 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:15.427557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 482/955] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:15.751243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4f315-1a0d-4cb1-914e-3e0c906b9461 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:15.751822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4f315-1a0d-4cb1-914e-3e0c906b9461 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:15.751822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4f315-1a0d-4cb1-914e-3e0c906b9461 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:15.753456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 474/956] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:15.758714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:15.759007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:15.759284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:15.778110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:15.779352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:15.854320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 483/957] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:15.876222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4f315-1a0d-4cb1-914e-3e0c906b9461 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:15.986259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4f315-1a0d-4cb1-914e-3e0c906b9461 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.030107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.030649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.030946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.031207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.031839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:16.033221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:16.042276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:16.043269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:16.044227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 475/958] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:16.052019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.113755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:16.158564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2223-f884-4c4c-beec-4b2c2a6ead25 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.192869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.193495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.193771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.194041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.194718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:16.195021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:16.196447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6baef8-deda-420c-b145-283877c0073c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:16.201692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 484/959] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:16.208873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.251750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:16.287642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9114d-264e-406b-ad5b-f6eebf360bc0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.287869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9114d-264e-406b-ad5b-f6eebf360bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.288129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9114d-264e-406b-ad5b-f6eebf360bc0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.289734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 485/960] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:16.295452 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff6bc-a3ab-439d-9e91-6c7d862e7dc0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.296020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff6bc-a3ab-439d-9e91-6c7d862e7dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.296519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff6bc-a3ab-439d-9e91-6c7d862e7dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.315407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-512ff6bc-a3ab-439d-9e91-6c7d862e7dc0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:16.316949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-512ff6bc-a3ab-439d-9e91-6c7d862e7dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:16.320687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67217ac-bc6e-4ac1-8ace-5753cac39a87 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.363201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.363875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.364153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.364421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.365104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:16.365402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:16.367327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:16.376651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 476/961] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:16.391560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 486/962] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:16.415189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.463224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f91097-dd9d-47ea-b0cf-5f633d56eac2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.463511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f91097-dd9d-47ea-b0cf-5f633d56eac2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.463835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f91097-dd9d-47ea-b0cf-5f633d56eac2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.465523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 487/963] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:16.470920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.471209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.471511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.490429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:16.491652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:16.541020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159d4fe-4c7c-41c9-bf5e-70e09bdaf60e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.544397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:16.565887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:16.581104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.581786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.582071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.582343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.586099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 488/964] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:16.597520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:16.602910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:16.603886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:16.604743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 477/965] 10.222.0.30 () {56 vars in 1120 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:16.610489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.612180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.726250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd99b5f-59f8-4081-ad30-857cf3c8ac64 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.749594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea71df4-c4ef-4722-ad27-ec5dbef49213 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.762053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.762767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.763052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.763322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.764074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:16.765523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:16.768204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING keystone.server.flask.common [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 05:01:16.782438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.sql.core [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 05:01:16.782438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 05:01:16.782438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [parameters: {'user_id': '3d7f92d2bce34b04906e7915ceae8b23', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 05:01:16.782438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80378) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Aug 05 05:01:16.791503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 05:01:16.792445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application yield resource Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application next(self.gen) Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise self.value Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application session.commit() Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 05:01:16.793500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.session.flush() Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application compat.raise_( Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 05:01:16.794973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 05:01:16.795969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application conn.query(q) Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application result.read() Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 05:01:16.796842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [parameters: {'user_id': '3d7f92d2bce34b04906e7915ceae8b23', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:16.798820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 05:01:16.800852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 489/966] 10.222.0.30 () {60 vars in 1131 bytes} [Fri Aug 5 05:01:16 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:16.803636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.810362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.812172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.812663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.813133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.814756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:16.815282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:16.817660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:16.829668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 478/967] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:16.834999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:16.859666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:16.862451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:16.910142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080a730-4d6b-404c-bdff-fdc41d5f8259 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.940984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8605f-9fdd-4aa8-bbdc-f862c2815f7b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:16.946699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.947522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.947967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.948404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.958255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:16.960552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:16.975487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:16.976714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 05:01:16.977881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:16.978973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:16.979752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:16.980200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:16.980560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 490/968] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:16.984123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:16.985980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.002272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.003474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:17.027142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:17.028460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:17.029560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:17.029560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:17.029560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:17.029560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:17.031416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 479/969] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:17.037737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.107021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ff020-c762-438c-a7c0-c72bbedacd5a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.118882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:17.140903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:17.155085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.156014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.156619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.157420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.158618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:17.159074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.161393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.166519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660bf78-d49e-497f-b437-e58bf60eedc4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.202688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.203466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.204229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.204229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.204995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:17.205402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.206886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.212162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 491/970] 10.222.0.30 () {56 vars in 1101 bytes} [Fri Aug 5 05:01:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:17.231666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 480/971] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:17.237297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.349432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6774c3-3c34-41b2-ab15-9d1c36a03a5c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.386467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.387489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.387949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.388378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.396171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.397816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0ec01486-a0ec-4c1f-a3c3-ecae45de9937 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:17.409987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:17.411142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 05:01:17.412359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:17.414592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 492/972] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:17.420568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.473029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7582e-7626-4cf6-9c8d-550c7ab9f814 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.473310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7582e-7626-4cf6-9c8d-550c7ab9f814 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.473586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7582e-7626-4cf6-9c8d-550c7ab9f814 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.475087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 493/973] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:17.479906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.480191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.480466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.498863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:17.500377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:17.523847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:17.529435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b596ba-074b-4add-9069-97382fad40f9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.566334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.567258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.567539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.567843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.568661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:17.568967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.570428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.574731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:17.580106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 494/974] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:17.591466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 481/975] 10.222.0.30 () {56 vars in 1149 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:17.592984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.596728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.694547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e064d0-5de8-4bd4-af71-f9140b8d7604 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.697907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead85c5f-e8fd-4613-a4a3-5c4ff3b5725e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.733040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.733693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.733972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.734241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.734946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.736052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.736358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.736724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.737004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.737222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:17.747445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:17.748473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:17.749314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:17.750037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 495/976] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:17.755380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.780425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:17.802684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:17.811513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.813225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.857038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7ca01-bcf8-47c8-a64d-3477ca539585 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:17.891358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:17.891981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:17.892262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:17.892566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:17.893252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:17.893528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:17.894903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:17.905371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 496/977] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:17.906653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:17.919500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 482/978] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:01:17.920570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.000992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.021876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38551a-ffb7-4c19-94c2-3c2f54c60e91 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.022836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.030446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7579d3-3450-4b50-9a72-5508dac88b03 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.060980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.061963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.062456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.062902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.063785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:18.064265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.065801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.065971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.066605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.066883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.067157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.071660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 497/979] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:18.077616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.080932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.082397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:18.099960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:18.100896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:18.101864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:18.101864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:18.101864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:18.101864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:18.101864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 483/980] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:18.106698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.186877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c4dbd-34be-46d3-a9c4-3dc5c7b023b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.209757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d80d7-6ab2-4af9-8c7f-c580141a0763 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.221457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.222087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.222366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.222645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.223434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] RBAC: Adding query filter params (interface=admin, service_id=1edb10db620e4963b958a5812abcf567, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:18.223744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.225141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.231810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 498/981] 10.222.0.30 () {56 vars in 1249 bytes} [Fri Aug 5 05:01:18 2022] GET /identity/v3/endpoints?service_id=1edb10db620e4963b958a5812abcf567&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:18.235902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.248463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.249022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.249298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.249567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.250250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:18.250556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.251994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.252613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.264544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 484/982] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:18.269915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.281980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.345439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2774b-38bc-47d4-bc4d-6b6a63947d08 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.379390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.380035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.380313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.380461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db3e31-7400-48b5-bbd3-5c5667df210e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.380636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.381245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.382614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.389480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 499/983] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:01:18 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:01:18.417927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.418653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.418925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.419201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.427782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.429339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-27c7fd8b-2f4f-43e2-91f1-ee4b799f5e50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:18.442291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:18.443314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:18.444301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:18.448276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 485/984] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:18.449793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.482076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b8cd4-b00d-4218-91b5-4d721dffac37 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.482422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b8cd4-b00d-4218-91b5-4d721dffac37 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.482792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b8cd4-b00d-4218-91b5-4d721dffac37 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.484315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 486/985] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:18.489782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.490097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.490452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.510028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:18.511545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:18.549103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.578382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.580417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342df32-d961-4b86-a8df-82a8159bfaa0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.586654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 487/986] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:18.608852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.618052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.619372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.619855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.620691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.621958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:18.622459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.624832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.641683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 500/987] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:18.647349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:18.708870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c053-fdc8-4725-8866-2ff8cd385ada None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.744408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3753d-d580-4e69-ae7d-e0268b873969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.745042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3753d-d580-4e69-ae7d-e0268b873969 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.745319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3753d-d580-4e69-ae7d-e0268b873969 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.745589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3753d-d580-4e69-ae7d-e0268b873969 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.746279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3753d-d580-4e69-ae7d-e0268b873969 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.747736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3753d-d580-4e69-ae7d-e0268b873969 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.756186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50395b-09ea-4dc8-93db-0c9c4792b4dd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:18.764400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 488/988] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:18 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:01:18.805222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:18.805862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:18.806138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:18.806406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:18.807223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] RBAC: Adding query filter params (role.id=5493cd3a48994597ab746dd76d27c6ea, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:18.807598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:18.809174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc12fc-8d81-4a8e-8bf9-6b92a694af68 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:18.811514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:18.824382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 501/989] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:18 2022] GET /identity/v3/role_assignments?user.id=8f6cb9dab5f246cabb4d8f1d7f61f20e&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=5493cd3a48994597ab746dd76d27c6ea => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:20.373948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c423b4-a61b-436f-958c-9715576aa073 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.374678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c423b4-a61b-436f-958c-9715576aa073 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.374678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c423b4-a61b-436f-958c-9715576aa073 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.375736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 489/990] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:20.381524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.382021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.382100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.401197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:20.402384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:20.480190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 502/991] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:20.497846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c423b4-a61b-436f-958c-9715576aa073 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:20.558411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:20.606515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c423b4-a61b-436f-958c-9715576aa073 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:20.641460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:20.642019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.642298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.642577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.643265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:20.644673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b728e1fa-4113-41ff-8fb1-b020c9e0af6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:20.653674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:20.654641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 05:01:20.655495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:20.662961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 490/992] 10.222.0.30 () {56 vars in 1126 bytes} [Fri Aug 5 05:01:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:20.663110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:20.763003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24d14d-ade2-4486-9aa3-aa9538aacffd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:20.772881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5fef9-e8e5-46b8-bdde-8bf545355c39 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.773750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5fef9-e8e5-46b8-bdde-8bf545355c39 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.774036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5fef9-e8e5-46b8-bdde-8bf545355c39 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.775387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 491/993] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:20.780792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b125a0a-9e38-4c13-a704-fb01fe6d8168 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.781152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b125a0a-9e38-4c13-a704-fb01fe6d8168 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.781426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b125a0a-9e38-4c13-a704-fb01fe6d8168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.791588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:20.800166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:20.800716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8b125a0a-9e38-4c13-a704-fb01fe6d8168 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:20.800826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.801072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.801346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.802266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8b125a0a-9e38-4c13-a704-fb01fe6d8168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:20.802444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:20.802694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:20.804092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:20.816363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 503/994] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:20.817399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:20.872060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 492/995] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:20.873387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a64487-ac3c-4141-8fe5-cb3987df0eff None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.873678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a64487-ac3c-4141-8fe5-cb3987df0eff None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.873954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a64487-ac3c-4141-8fe5-cb3987df0eff None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.875192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 493/996] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:20.880347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.880633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.880912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.898310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:20.899434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:20.916450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a231e-fef5-4b1e-b727-c232640f095c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:20.945149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:20.950558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:20.951207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:20.951485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:20.951758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:20.952450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:20.953039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:20.954115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:20.958954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 504/997] 10.222.0.30 () {56 vars in 1122 bytes} [Fri Aug 5 05:01:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:20.959710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:20.966892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 494/998] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:20.967631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.022172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:21.065978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb721eaa-7ce1-4c1d-9ea0-7a6e0557a2ea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.068909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ccf6-99f6-4c26-a930-7af4b56448c6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.099450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.100123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.100412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.100682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.100859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.101303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.101476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.101755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.102024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.102720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.102815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.104090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:21.111612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:21.113030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:21.113998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 505/999] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:21.114801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.124251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 495/1000] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:21.125299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.180510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:21.227579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134a763-6380-424d-86d5-c16d4cbc5ba2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.250164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:21.253305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515317-e65a-4879-8d4a-32f4029560bb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.266024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.266711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.266986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.267291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] PATH_INFO: `/v3/services/image` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.267966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.269368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:21.278599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:21.279615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:21.285738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 506/1001] 10.222.0.30 () {56 vars in 1105 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:21.286541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.304708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.305329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.305612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.305885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.306571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:21.306873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.308306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.313941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 496/1002] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:21.320228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.397313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c83cb-66c2-453a-8f14-e8038fabe06f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.422209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:21.426091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2d834-bdad-48ea-bf1f-92bbaf4aad1b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.435251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.435837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.436122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.436393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.437073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:21.437379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.438736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.443785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 507/1003] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:21.449313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.468172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.468800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.469112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.469353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.470024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:21.470325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.471693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.476737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 497/1004] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:21.482421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.499608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:21.553666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1fcd-b991-4ac5-be8d-e6f7bf502a89 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.587031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cf8ce-a060-43a8-826e-c4c0765ab4ea None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.590562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.591194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.591478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.591753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.592457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] RBAC: Adding query filter params (type=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:21.592769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.594153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.606505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 508/1005] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:21.624111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.624870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.625152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.625424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.626525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] RBAC: Adding query filter params (interface=public, service_id=66ec6f2ea76d47b58fba26ede13ab1aa, region_id=RegionOne) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:21.626838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.628297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f2ce8-b16c-43fe-8553-5ecc236bc4bb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.635962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 498/1006] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/endpoints?service_id=66ec6f2ea76d47b58fba26ede13ab1aa&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:21.640967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:21.754043 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81291c-1d2c-4a55-bcaa-f37ce5a1cbfe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:21.767729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:21.789744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de68b8-a77e-4280-9772-87d95dd10784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:21.790376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de68b8-a77e-4280-9772-87d95dd10784 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:21.790661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de68b8-a77e-4280-9772-87d95dd10784 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:21.790935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de68b8-a77e-4280-9772-87d95dd10784 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:21.791597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de68b8-a77e-4280-9772-87d95dd10784 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:21.793012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de68b8-a77e-4280-9772-87d95dd10784 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:21.799490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 509/1007] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:01:21 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:01:23.121661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220e13-6a26-47a5-a721-59b387bc4ae3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.122228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220e13-6a26-47a5-a721-59b387bc4ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.122228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220e13-6a26-47a5-a721-59b387bc4ae3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.123825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 499/1008] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:23.128846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.129277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.129354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.147992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:23.149174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:23.219432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 510/1009] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:23.241787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220e13-6a26-47a5-a721-59b387bc4ae3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:23.272570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:23.347594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220e13-6a26-47a5-a721-59b387bc4ae3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:23.381249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986d9ca-f7d9-41a0-9c32-7e76a61e6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:23.381890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d9ca-f7d9-41a0-9c32-7e76a61e6600 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.382175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d9ca-f7d9-41a0-9c32-7e76a61e6600 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.382444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d9ca-f7d9-41a0-9c32-7e76a61e6600 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.383598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986d9ca-f7d9-41a0-9c32-7e76a61e6600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:23.385060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986d9ca-f7d9-41a0-9c32-7e76a61e6600 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:23.386858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 500/1010] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:23.392656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:23.495406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fdd03-8bbf-4258-8383-43ac4dec92ea None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:23.502753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:23.529830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254bb78-f73a-4bec-b46b-294bb0209735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:23.530372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254bb78-f73a-4bec-b46b-294bb0209735 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.530650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254bb78-f73a-4bec-b46b-294bb0209735 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.530920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254bb78-f73a-4bec-b46b-294bb0209735 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.531601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254bb78-f73a-4bec-b46b-294bb0209735 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:23.533064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254bb78-f73a-4bec-b46b-294bb0209735 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:23.567798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 511/1011] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:23 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:23.574676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e65e-71b6-4223-913f-efc4c6d08629 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.574975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e65e-71b6-4223-913f-efc4c6d08629 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.575258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e65e-71b6-4223-913f-efc4c6d08629 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.576927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 501/1012] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:23.582634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f047ed-3c57-4fb3-a547-a95956951986 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.582988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f047ed-3c57-4fb3-a547-a95956951986 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.583268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f047ed-3c57-4fb3-a547-a95956951986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.602563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-86f047ed-3c57-4fb3-a547-a95956951986 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:23.603813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-86f047ed-3c57-4fb3-a547-a95956951986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:23.675265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 512/1013] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:23.698259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e65e-71b6-4223-913f-efc4c6d08629 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:23.784840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be66b0b-061e-472b-92b3-daadab4ce729 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.785092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be66b0b-061e-472b-92b3-daadab4ce729 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.785314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be66b0b-061e-472b-92b3-daadab4ce729 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.786967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 513/1014] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:23.792661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.792943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.793214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.805550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:23.812334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e65e-71b6-4223-913f-efc4c6d08629 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:23.820136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:23.821525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:23.846589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:23.847217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:23.847501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:23.848095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:23.848781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:23.850142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:23.867798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 502/1015] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:23 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:23.901449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 514/1016] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:23 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:23.924808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:24.048558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09c4e-d3fc-4987-8498-d7dfbff60887 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:24.087203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:24.087900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:24.088194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:24.088472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:24.089117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:24.090534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:24.100098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:24.101096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:24.103173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 503/1017] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:24.110278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:24.126991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:24.223075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d1657-adaf-40c2-a1de-71114dcc8ef1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:24.257759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:24.258384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:24.258664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:24.258936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:24.259602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:24.259931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:24.261307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:24.266514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 515/1018] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:24.272312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:24.295345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:24.382995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807c1a-66c3-47d9-acb4-1f470072e7fb None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:24.417975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:24.418647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:24.418927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:24.419202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:24.419904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:24.420216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:24.421603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ef539-edfb-468e-a40e-e78683acb812 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:24.427552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 504/1019] 10.222.0.30 () {56 vars in 1112 bytes} [Fri Aug 5 05:01:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:24.433092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:24.514554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:24.536928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f95e9-7b91-43a1-82ff-e2638217a976 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:24.571940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd2b08-aa25-4d2c-9689-e04c883dd8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:24.572402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2b08-aa25-4d2c-9689-e04c883dd8fb None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:24.572681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2b08-aa25-4d2c-9689-e04c883dd8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:24.572946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2b08-aa25-4d2c-9689-e04c883dd8fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:24.573675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd2b08-aa25-4d2c-9689-e04c883dd8fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:24.575089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd2b08-aa25-4d2c-9689-e04c883dd8fb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:24.596302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 516/1020] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:25.608794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a68119-7742-4de1-80d9-59937cc46ad9 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:25.609349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a68119-7742-4de1-80d9-59937cc46ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:25.609349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a68119-7742-4de1-80d9-59937cc46ad9 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:25.610854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 505/1021] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:25.616046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:25.616448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:25.616635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:25.635081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:25.636289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:25.711798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 517/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:25.736020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a68119-7742-4de1-80d9-59937cc46ad9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:25.824214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:25.842954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a68119-7742-4de1-80d9-59937cc46ad9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:25.877241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807faf2-9659-4258-a7ab-76fe09df24c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:25.877811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807faf2-9659-4258-a7ab-76fe09df24c8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:25.878090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807faf2-9659-4258-a7ab-76fe09df24c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:25.878364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807faf2-9659-4258-a7ab-76fe09df24c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:25.879524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807faf2-9659-4258-a7ab-76fe09df24c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:25.880992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807faf2-9659-4258-a7ab-76fe09df24c8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:25.882737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 506/1023] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:25.888523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:25.895641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac355-61b1-4317-9184-5f24454380b1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:25.895974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac355-61b1-4317-9184-5f24454380b1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:25.896254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac355-61b1-4317-9184-5f24454380b1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:25.897707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 507/1024] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:25.902369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:25.902743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:25.903003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:25.920852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:25.921982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:25.953985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:25.992962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 508/1025] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:25.999090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eda0f-1c34-49e8-a976-bf95e3ddcf02 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.021467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:26.036293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.036932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.037217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.037487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.038184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:26.039631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:26.071792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 518/1026] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:01:25 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:26.120790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:26.143871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6e747-1748-4814-9c3b-3cfae113bd84 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.183098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.183815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.184196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.184471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.185201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:26.186609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:26.196787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:26.197882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:26.198769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:26.200229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 509/1027] 10.222.0.30 () {56 vars in 1105 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:26.204605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:26.311066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9313d-887b-47a3-ab8c-ac22e0a642fe None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.327839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:26.347111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.348049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.348501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.348957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.349782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] RBAC: Adding query filter params (name=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:26.350238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:26.351752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:26.357400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 519/1028] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:26.363271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:26.464912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab74a-868c-4b9d-99b5-a9d3d896d3f5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.500085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.500706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.500966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.501261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.501923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] RBAC: Adding query filter params (type=image) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:26.502226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:26.503598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:26.513471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 510/1029] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:26.514723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:26.618433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300011d-42c1-4490-81a9-8ef4d21ac3ff None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.653206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.653874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.654137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.654431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.655169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] RBAC: Adding query filter params (interface=public, service_id=ba4312f113844dffacb76d5a932fe1b5, region_id=RegionOne) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:26.655481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:26.656880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db355446-dc27-4dbc-a145-e0df74e1fc42 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:26.662420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 520/1030] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/endpoints?service_id=ba4312f113844dffacb76d5a932fe1b5&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:26.667985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:26.673017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01e9e3-2698-4d3f-ba26-c9139a6ee513 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.673305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01e9e3-2698-4d3f-ba26-c9139a6ee513 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.673584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01e9e3-2698-4d3f-ba26-c9139a6ee513 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.675007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 521/1031] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:26.680292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.680590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.680864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.681951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:26.700619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:26.701955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:26.751225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:26.775474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f32d07-3c51-4615-bc2f-ee81e820a00e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.780688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 522/1032] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:26.804248 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:26.810744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.811384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.811660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.812033 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.812682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:26.814052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:26.820960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 511/1033] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:01:26.920175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732a125-2ba1-4f5c-8623-57b6fc58d003 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:26.958014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:26.958685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:26.958926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:26.959216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:26.970622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:26.975500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:26.976484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:26.977355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 523/1034] 10.222.0.30 () {56 vars in 1120 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:26.983305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:27.069499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:27.095334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62bae-fff1-4f84-92b8-607dc4bb1cb6 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:27.129403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:27.130029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:27.130311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:27.130592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:27.131289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:27.131610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:27.133025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:27.141692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 512/1035] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:01:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:27.147283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:27.182427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:27.251915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477adb1-d6ff-4f33-8925-43bcc0fa1319 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:27.286912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:27.287566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:27.287870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:27.288143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:27.298693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:27.300177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:27.315433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:27.316536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:27.317525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:27.317525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:27.317525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:27.317525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:27.317525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 524/1036] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:27.322488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:27.418998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2b0f-562a-4a49-84be-d7c3a6f436d2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:27.452107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:27.452724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:27.453003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:27.453232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:27.453950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:27.454233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:27.455608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:27.457989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:27.468584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 513/1037] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:27.474199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:27.560480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:27.576840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a45751-8ce8-4ab5-bee5-f2fdff2c99cd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:27.611538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:27.612209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:27.612489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:27.612759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:27.620341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:27.621838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:27.632864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:27.633789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:27.634771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:27.638833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 525/1038] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:01:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:27.640300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:27.745585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59372f1-c67b-4943-ac94-f69dabed44b5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:27.780618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:27.781236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:27.781517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:27.781789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:27.782580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:27.782886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:27.784338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a3d1a-a1ec-4e1f-bcc5-427c8e54e9e8 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:27.806490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 514/1039] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:01:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:27.808143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:27.906751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a0810-7864-4e81-b23c-8bcb6346dcbf None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:27.940954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:27.941591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:27.941875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:27.942150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:27.960812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.022216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.023696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.119036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf09b9-a08b-476d-8eff-05d4337dc3f5 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.119438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf09b9-a08b-476d-8eff-05d4337dc3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.119784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf09b9-a08b-476d-8eff-05d4337dc3f5 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.121425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 515/1040] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:28.126156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.126441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.126713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.132636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 526/1041] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:01:28.133490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.146412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:28.147609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:28.182701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.197779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.220125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 516/1042] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:28.239627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.242698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae442d37-e18c-42a5-b023-6448531f2b22 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.276242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.277178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.277697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.278169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.288592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.290276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:28.306661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:28.307608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:28.308536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:28.308536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:28.308536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:28.308536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:28.311982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 527/1043] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:28.313516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.343455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df427f37-7393-4a27-a288-f0fba72c43a8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.378503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.379156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.379434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.379710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.392571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:28.397457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:28.398403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:28.399229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 517/1044] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:28.404669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.413391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.418131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c839-be05-4cf6-a32b-bdb1e8ff0a4b None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.454118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.454663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.454935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.455033 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.455216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.455940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:28.456260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.457644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.469726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 528/1045] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:28.475335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.517349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4b2dd-938b-4f74-abbc-1c85dba8f047 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.552541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.553169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.553449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.553723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.554440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:28.554774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.556175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.564893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 518/1046] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:28.570474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.584330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd9754-5a55-4d54-97aa-8f12136a965d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.620448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.621193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.621450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.621700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.628473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.629858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:28.642228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:28.643108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:28.644052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 529/1047] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:28.649163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.669415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.680174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc8455-25c0-4002-bce7-2d6490aab475 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.688036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.718554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.719136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.720003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.720275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] PATH_INFO: `/v3/users/35f8ea956bd74578946add21ae581f21` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.738813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] RBAC: Authorizing `identity:get_user(user_id=35f8ea956bd74578946add21ae581f21)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.740691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.747561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 519/1048] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/users/35f8ea956bd74578946add21ae581f21 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:28.748598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.754767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096242bb-b04c-4913-aecb-4d0aa996ee38 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.790040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.790667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.790951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.791223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.792009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:28.792317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.793822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.806555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 530/1049] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:28.812535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:28.856161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507932bf-0d6d-4597-8f7e-dbd3fa37a770 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.888501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.889121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.889395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.889668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.895262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.917772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e1d-fbed-4d43-affe-e3a91b841967 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:28.925883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:28.953641 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:28.954257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.954531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.954803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.955580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] RBAC: Adding query filter params (role.id=5493cd3a48994597ab746dd76d27c6ea, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=5134c03bb8854cd4bee58644e6eacc51) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:28.955918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.957345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc90e6-5ac0-4cb6-a794-f2a0348a830b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.962960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 531/1050] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/role_assignments?user.id=5134c03bb8854cd4bee58644e6eacc51&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=5493cd3a48994597ab746dd76d27c6ea => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:28.963319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:28.964183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531381-12bf-436c-b1d4-4ddb7d197fb1 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.964466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531381-12bf-436c-b1d4-4ddb7d197fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.964734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531381-12bf-436c-b1d4-4ddb7d197fb1 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:28.964833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:28.966146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 532/1051] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:28.972001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:28.972196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:28.972474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.000266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:29.001603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:29.083057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 533/1052] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:29.090723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 520/1053] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:01:29.098992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:29.107361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:29.169966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:29.174034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:29.207570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0fe18-bc2c-4e0e-9735-3fc7faf01f2c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:29.215633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dc638-d22f-42d1-ba82-a31c6acaf5fa None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:29.248915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:29.249669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:29.250047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:29.250418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.251824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] RBAC: Authorizing `identity:get_project(project_id=403d37853e1f4e059b26eb93ab397401)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:29.252345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:29.253004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:29.253286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:29.253389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:29.253551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] PATH_INFO: `/v3/services/image` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.254183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:29.255552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:29.259928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 534/1054] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:29 2022] GET /identity/v3/projects/403d37853e1f4e059b26eb93ab397401 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:29.261035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:29.271022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:29.272099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:29.273001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 521/1055] 10.222.0.30 () {56 vars in 1105 bytes} [Fri Aug 5 05:01:29 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:29.278358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:29.372722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ae4e-3630-4496-8e1b-e0d282341d82 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:29.380331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fface03c-95cd-49e4-a38e-cec4ded40535 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:29.406274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:29.415584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:29.415805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:29.416239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:29.416343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:29.416521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:29.416634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:29.416795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.416890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.417566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] RBAC: Adding query filter params (name=image) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:29.417676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=403d37853e1f4e059b26eb93ab397401, user.id=35f8ea956bd74578946add21ae581f21) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:29.417873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:29.417968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:29.419294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:29.419399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:29.435815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 535/1056] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:29 2022] GET /identity/v3/role_assignments?user.id=35f8ea956bd74578946add21ae581f21&scope.project.id=403d37853e1f4e059b26eb93ab397401&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:29.436063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 522/1057] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:29.436172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:29.440783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:29.548474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4c752-f9e9-4cc0-907e-9a0f50ff1cf0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:29.585447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:29.586083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:29.586362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:29.586637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.587317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:29.587624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:29.589033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815db7dd-9c83-4674-9c49-ec852403b9f4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:29.595006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 536/1058] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:29.601109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:29.700685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db0f6a-68fb-4a2f-8e12-b78c99a2291b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:29.734675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57418ec-9b45-4a96-b02f-8ca9daf465f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:29.735300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57418ec-9b45-4a96-b02f-8ca9daf465f5 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:29.735608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57418ec-9b45-4a96-b02f-8ca9daf465f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:29.735902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57418ec-9b45-4a96-b02f-8ca9daf465f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:29.736631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57418ec-9b45-4a96-b02f-8ca9daf465f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:29.737995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57418ec-9b45-4a96-b02f-8ca9daf465f5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:29.758360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 523/1059] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:01:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:01:30.925049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ffbaf-3dc8-4262-a122-5a9cd7a00769 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:30.925620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ffbaf-3dc8-4262-a122-5a9cd7a00769 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:30.925620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ffbaf-3dc8-4262-a122-5a9cd7a00769 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:30.927282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 537/1060] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:30.932740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:30.933156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:30.933292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:30.953011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:30.954206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:31.005820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.037645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 524/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:31.059955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ffbaf-3dc8-4262-a122-5a9cd7a00769 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.131109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.167373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ffbaf-3dc8-4262-a122-5a9cd7a00769 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.202474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.203051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.203325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.203599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.213910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:31.218935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:31.225091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:31.225981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 538/1062] 10.222.0.30 () {56 vars in 1120 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:31.226688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.325674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8daea0-9ea9-47ce-99ff-a33b587c5fa8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.359835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.360473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.360750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.361023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.361745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:31.362057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:31.363445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffba9c8-b80d-4b09-83d6-5a1d6b1b4366 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:31.376174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 525/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:31.377481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.380995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40dd37-0955-4001-b768-d7a07c85338c None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.381289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40dd37-0955-4001-b768-d7a07c85338c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.381556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40dd37-0955-4001-b768-d7a07c85338c None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.382998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 526/1064] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:01:31.387652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.387967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.388246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.406276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:31.407283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:31.438522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.479706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 527/1065] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:31.486645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d8e9c-9aa9-4c6a-a85c-2301fc1c137e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.502393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.515041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.524636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.525700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.526122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.526522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.539870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:31.541328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:31.559225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:31.560169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:31.561052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:31.561052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:31.561052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:31.561052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:31.561294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 539/1066] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:31.566584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.607614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992bd3c-acfd-44f9-8747-26ba2dbbc587 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.642433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.643081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.643359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.643629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.654303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:31.659172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:31.660255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:31.661101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 528/1067] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:31.666268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.675341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb87d51-0a40-4501-9d4a-009de633e3f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.696340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.710843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.711483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.711787 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.712061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.712763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:31.713073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:31.714458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:31.726716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 540/1068] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:31.732871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.775581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.782302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29a059-4fe6-4dfd-a24e-92ba11d46d2f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.816363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.816989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.817271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.817553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.818263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:31.818574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:31.820009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad2d67-de0f-4aed-ba38-c47540204bd0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:31.828184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 529/1069] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:31.829335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fc9b7-cd84-491c-baa2-70cdf4d3cfcf None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.829616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fc9b7-cd84-491c-baa2-70cdf4d3cfcf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.829888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fc9b7-cd84-491c-baa2-70cdf4d3cfcf None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.831232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 530/1070] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:31.835615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4376ae-af19-4b54-90d0-cb3b53fe521e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.835792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fc9b7-cd84-491c-baa2-70cdf4d3cfcf None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:31.871481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:31.872154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.872432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.872701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.879691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:31.881196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ddc36fa0-30f2-4dc4-8d8b-eea27b3b97e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:31.892864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:31.893942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:31.895136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 541/1071] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:31.895897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:31.895897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:31.895897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:31.913549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:31.914738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:31.928626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:31.949918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fc9b7-cd84-491c-baa2-70cdf4d3cfcf None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:31.982218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 542/1072] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:31.983122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.002107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.002796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.003072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.003342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] PATH_INFO: `/v3/users/35f8ea956bd74578946add21ae581f21` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.004934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] RBAC: Authorizing `identity:get_user(user_id=35f8ea956bd74578946add21ae581f21)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.006545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.008328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 531/1073] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/users/35f8ea956bd74578946add21ae581f21 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:32.009076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.014032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.086926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ad02d-4ed4-4600-8a36-17979abef028 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.109447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53566357-8d16-4305-b4c2-df6bf33c558a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.122002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.122668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.122946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.123215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.124062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:32.124368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.125839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.142568 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 543/1074] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:01:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:32.143476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.143615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.144144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.144540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.144817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.170081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.213197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.214645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.230509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.243182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780bb63-52e0-4d31-85cc-f9b8410efb27 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.277853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.278509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.278785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.279052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.280322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] RBAC: Authorizing `identity:get_project(project_id=403d37853e1f4e059b26eb93ab397401)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.281779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.283463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 544/1075] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/projects/403d37853e1f4e059b26eb93ab397401 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:32.284234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.315005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 532/1076] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:32.315961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.371353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.390669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87950-15f9-40fc-a567-dc7b596062d0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.413809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d89a5c-e2f4-48da-9a69-39d8fec867ba None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.425631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.426286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.426918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.427179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.437229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.438691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.448383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.449059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.449340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.449672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401/users/35f8ea956bd74578946add21ae581f21/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.451747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] RBAC: Authorizing `identity:create_grant(project_id=403d37853e1f4e059b26eb93ab397401, user_id=35f8ea956bd74578946add21ae581f21, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:32.453118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:32.453995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:32.454909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:32.454909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:32.454909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:32.454909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:32.455140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.455211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 545/1077] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:32.455211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.467078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.483815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 533/1078] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:01:32 2022] PUT /identity/v3/projects/403d37853e1f4e059b26eb93ab397401/users/35f8ea956bd74578946add21ae581f21/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:01:32.483815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.567488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c622e-c8d2-4029-99c4-61d5129f713a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.586796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaa74a-d68d-4c93-ac20-66b11409a585 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.605910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.606267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.606548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.606896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.607192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.619523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.621560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.630820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.631458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.631739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.632048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.633720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:32.633720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.634413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:32.639200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:32.640135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:32.641181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:32.641181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:32.641181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:32.641181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:32.641181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 546/1079] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:32.647866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.652021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 534/1080] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:32.657397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.711626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.757074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d734bf6-0c5d-46e0-86c2-5e8930e66f26 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.761713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff91b59-237a-417b-a202-1c3d916df1d3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.796041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.796704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.796984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.797261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.799055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.799637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.799951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.800373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.800909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:32.801216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.802939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.804832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.806266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.819885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 547/1081] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:32.820070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:32.821097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:32.822203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 535/1082] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:01:32.825802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:32.930525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee81d-438a-4129-a069-137ec7265309 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:32.952639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:32.969497 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:32.970141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:32.970426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:32.970699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:32.971475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:32.971790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:32.973272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893c958-7578-4ea6-9815-72bd0a87b484 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:32.986124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 548/1083] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:32.992914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:33.038447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:33.108313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a00c3-c5d2-4340-83d5-726b7ff67fdd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:33.143895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50548c5-9b40-4533-8b83-d4afb0733f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:33.144578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50548c5-9b40-4533-8b83-d4afb0733f79 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:33.144856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50548c5-9b40-4533-8b83-d4afb0733f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:33.145126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50548c5-9b40-4533-8b83-d4afb0733f79 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/5134c03bb8854cd4bee58644e6eacc51/roles/5493cd3a48994597ab746dd76d27c6ea` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:33.147251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50548c5-9b40-4533-8b83-d4afb0733f79 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=5134c03bb8854cd4bee58644e6eacc51, role_id=5493cd3a48994597ab746dd76d27c6ea)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:33.148712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50548c5-9b40-4533-8b83-d4afb0733f79 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:33.162859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 536/1084] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:01:32 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/5134c03bb8854cd4bee58644e6eacc51/roles/5493cd3a48994597ab746dd76d27c6ea => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:01:34.453348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943af275-69a3-4be8-86b6-bc46ae4c143b None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:34.453912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943af275-69a3-4be8-86b6-bc46ae4c143b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:34.453912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943af275-69a3-4be8-86b6-bc46ae4c143b None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:34.455279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 549/1085] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:34.460618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:34.461023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:34.461171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:34.480134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:34.481327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:34.551238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 537/1086] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:34.574615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943af275-69a3-4be8-86b6-bc46ae4c143b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:34.679026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943af275-69a3-4be8-86b6-bc46ae4c143b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:34.714433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:34.715029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:34.715311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:34.715578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:34.725919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:34.730821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:34.732071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:34.733133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 550/1087] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:34.738257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:34.765270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:34.842662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d60b3a-f681-43e2-8098-57ad7ff9d210 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:34.877238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:34.877865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:34.878147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:34.878427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:34.879141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:34.879444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:34.880848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:34.889299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 538/1088] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:34.895276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:34.943401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.019145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd09ba-1294-4379-b803-acaaf85b030c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.053259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca18e9-1d39-4e15-97f3-c68d1de3973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.053898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca18e9-1d39-4e15-97f3-c68d1de3973b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.054176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca18e9-1d39-4e15-97f3-c68d1de3973b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.054445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca18e9-1d39-4e15-97f3-c68d1de3973b None admin] PATH_INFO: `/v3/users/35f8ea956bd74578946add21ae581f21` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.055660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca18e9-1d39-4e15-97f3-c68d1de3973b None admin] RBAC: Authorizing `identity:get_user(user_id=35f8ea956bd74578946add21ae581f21)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.057127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca18e9-1d39-4e15-97f3-c68d1de3973b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.058776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 551/1089] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:34 2022] GET /identity/v3/users/35f8ea956bd74578946add21ae581f21 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:35.064909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.098879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa0e25-bbc9-48c4-a0fa-ce9108d4d813 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.099159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa0e25-bbc9-48c4-a0fa-ce9108d4d813 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.099420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa0e25-bbc9-48c4-a0fa-ce9108d4d813 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.100812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 552/1090] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:35.105571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.105856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.106133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.124443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:35.125646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:35.153788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.168591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5af976-ee18-4b4c-97d2-96ce93458222 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.193001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 553/1091] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:35.201630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.202254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.202540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.202814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.216292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.255815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.275922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.277419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.321372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53830de7-66fe-4a3f-86b6-33146350e183 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.359309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.359993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.360278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.360555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.368289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.372264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:35.377125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:35.378246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:35.380701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 554/1092] 10.222.0.30 () {56 vars in 1120 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:35.384500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.385092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 539/1093] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:01:35.390754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.477761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.481575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df91ced-0b89-437b-a481-5648d3350893 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.489065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc27f7-acae-4839-b0f0-c703470d73cc None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.514546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.515094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.515372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.515642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.516381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:35.516686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.518057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.521629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.522272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.522555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.522835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.524093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] RBAC: Authorizing `identity:get_project(project_id=403d37853e1f4e059b26eb93ab397401)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.525514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.525803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 555/1094] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:35.527810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 540/1095] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/projects/403d37853e1f4e059b26eb93ab397401 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:35.531573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.532985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.586411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.633773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243ec74-fde5-41ff-b3eb-4c5410574e2d None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.649935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719be63-07d6-4e0c-a9c9-7ac239a065bd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.669080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.669697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.669975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.670247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.671018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=403d37853e1f4e059b26eb93ab397401, user.id=35f8ea956bd74578946add21ae581f21) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:35.671327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.672797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.679170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 541/1096] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/role_assignments?user.id=35f8ea956bd74578946add21ae581f21&scope.project.id=403d37853e1f4e059b26eb93ab397401&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:35.686438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.687114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.687392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.687663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.700403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.701984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:35.718121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:35.719115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:35.720091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:35.720091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:35.720091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:35.720091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:35.720091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 556/1097] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:35.725941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.835750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972aa78-0e57-4aaf-b6b8-ae69922a87cf None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:35.857361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.872037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:35.872664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:35.872942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:35.873211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:35.873912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:35.874211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:35.875586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:35.888083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 542/1098] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:35.893734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:35.910718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:35.996721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e191a-4f1c-47f7-b504-f39dc15145fd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:36.031965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:36.032608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:36.032894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:36.033152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:36.041036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:36.042475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:01:36.054261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:36.055261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:36.056249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 557/1099] 10.222.0.30 () {56 vars in 1093 bytes} [Fri Aug 5 05:01:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:36.061467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:36.163701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb68a3-35e2-4d10-8a7e-b06d91d995ec None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:36.197762 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:36.198397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:36.198682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:36.198960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:36.199884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:36.200199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:36.201630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:36.218579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 543/1100] 10.222.0.30 () {56 vars in 1133 bytes} [Fri Aug 5 05:01:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:36.224453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:36.294351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:36.326649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7e033-9f0c-4f95-b70b-7af4204c2f36 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:36.361038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:36.361661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:36.361947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:36.362219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:36.431053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:36.432459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:36.501328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:36.534994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 558/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:36.540975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:36.582237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:36.642630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af285f6-d81b-4afa-8f96-917af0c9fe26 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:36.677770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2859-af53-4615-81bf-788289f84385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:36.678413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2859-af53-4615-81bf-788289f84385 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:36.678699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2859-af53-4615-81bf-788289f84385 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:36.678976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2859-af53-4615-81bf-788289f84385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:36.689032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e2859-af53-4615-81bf-788289f84385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:36.690425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e2859-af53-4615-81bf-788289f84385 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-104e2859-af53-4615-81bf-788289f84385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:01:36.705689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:36.706773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:36.712356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:01:36.712356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:01:36.712356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:01:36.712356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:36.712356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 544/1102] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:36.712759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:36.816011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec65f8-ef56-458d-8fad-26d2bf88fc93 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:36.850796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:36.851405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:36.851687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:36.851983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:36.852661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:36.852967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:36.854324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:36.865605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 559/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:36.871492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2859-af53-4615-81bf-788289f84385 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:36.965511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:36.976117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2859-af53-4615-81bf-788289f84385 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:37.012515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:37.013203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:37.013451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:37.013751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:37.020921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:37.022372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:01:37.033777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:37.034697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:01:37.035603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 545/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:37.041051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:37.076128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:37.146811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86750bf-db64-4af6-a83a-b3f4e23f6f10 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:37.181516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:37.182129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:37.182408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:37.182684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:37.183434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:37.183821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:37.185292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88babe-97ac-47f1-9d6d-2cd7090ddbb3 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:37.198161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 560/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:37.204518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:37.309189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9038a-1bc4-4f3f-8783-664cd4bcae50 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:37.343308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:37.343952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:37.344226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:37.344496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:37.345258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] RBAC: Adding query filter params (role.id=5493cd3a48994597ab746dd76d27c6ea, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=5134c03bb8854cd4bee58644e6eacc51) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:37.345582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:37.346978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d39df4-27b2-4e2c-99f9-e1f60a803dab None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:37.353136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 546/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:37 2022] GET /identity/v3/role_assignments?user.id=5134c03bb8854cd4bee58644e6eacc51&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=5493cd3a48994597ab746dd76d27c6ea => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:37.685736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89f4d3-8f56-45e1-99e4-823cfd27bee2 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:37.686297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89f4d3-8f56-45e1-99e4-823cfd27bee2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:37.686297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89f4d3-8f56-45e1-99e4-823cfd27bee2 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:37.687627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 561/1107] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:37.692982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244150-50d3-49f7-a215-4229de90259d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:37.693442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244150-50d3-49f7-a215-4229de90259d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:37.693530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244150-50d3-49f7-a215-4229de90259d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:37.711955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fe244150-50d3-49f7-a215-4229de90259d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:37.713127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fe244150-50d3-49f7-a215-4229de90259d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:37.723013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:37.787235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 547/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:37.812081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89f4d3-8f56-45e1-99e4-823cfd27bee2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:37.909099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:37.919339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89f4d3-8f56-45e1-99e4-823cfd27bee2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:37.952805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177cd78-3c2e-4393-8749-aeb39adac588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:37.953383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177cd78-3c2e-4393-8749-aeb39adac588 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:37.953656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177cd78-3c2e-4393-8749-aeb39adac588 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:37.953922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177cd78-3c2e-4393-8749-aeb39adac588 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:37.955049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177cd78-3c2e-4393-8749-aeb39adac588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:37.956512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177cd78-3c2e-4393-8749-aeb39adac588 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:37.958098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 562/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:37.964243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244150-50d3-49f7-a215-4229de90259d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:38.067090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244150-50d3-49f7-a215-4229de90259d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:38.100606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af080e-9228-45bc-abbd-26537062c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:38.101234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af080e-9228-45bc-abbd-26537062c4a6 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:38.101519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af080e-9228-45bc-abbd-26537062c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:38.101800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af080e-9228-45bc-abbd-26537062c4a6 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:38.102484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af080e-9228-45bc-abbd-26537062c4a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:38.103956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af080e-9228-45bc-abbd-26537062c4a6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:38.122213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:38.127694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 548/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:01:37 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:40.151453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb6f25-aa3f-46d4-83f2-aab0021d8c23 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:40.152605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb6f25-aa3f-46d4-83f2-aab0021d8c23 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:40.152605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb6f25-aa3f-46d4-83f2-aab0021d8c23 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:40.153858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 563/1111] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:40.159365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:40.159833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:40.159927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:40.179976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:40.181168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:40.251878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 549/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:01:40.284820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb6f25-aa3f-46d4-83f2-aab0021d8c23 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:40.394246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb6f25-aa3f-46d4-83f2-aab0021d8c23 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:40.442376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:40.443029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:40.443311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:40.443588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] PATH_INFO: `/v3/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:40.445122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] RBAC: Authorizing `identity:get_role(role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:40.446514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:40.447892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 564/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:40 2022] GET /identity/v3/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:40.454195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:40.560127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42e0f2-633a-4fc4-a7d1-d51e21a9d18f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:40.588968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:40.595477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:40.596142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:40.596403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:40.596677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] PATH_INFO: `/v3/users/8573f95032444cceb9d70c7fdff22f68` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:40.611207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] RBAC: Authorizing `identity:get_user(user_id=8573f95032444cceb9d70c7fdff22f68)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:40.612675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:40.614280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 550/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:40 2022] GET /identity/v3/users/8573f95032444cceb9d70c7fdff22f68 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:40.620139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:40.649216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:40.727548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fbc75-c3b7-4fd1-b6d3-a465acdd10cd None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:40.761977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:40.762608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:40.762846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:40.763142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:40.832126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:40.833517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:40.863821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:40.940336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 565/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:40.946548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:41.054215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ecd08-b59b-489a-8321-cfaf4dc2683c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:41.088950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b00fa5-04e6-46ac-a521-48283f4fd9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:41.089509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b00fa5-04e6-46ac-a521-48283f4fd9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:41.089785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b00fa5-04e6-46ac-a521-48283f4fd9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:41.090108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b00fa5-04e6-46ac-a521-48283f4fd9e5 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:41.091369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b00fa5-04e6-46ac-a521-48283f4fd9e5 None admin] RBAC: Authorizing `identity:get_project(project_id=403d37853e1f4e059b26eb93ab397401)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:41.092838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b00fa5-04e6-46ac-a521-48283f4fd9e5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:41.094478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 551/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:40 2022] GET /identity/v3/projects/403d37853e1f4e059b26eb93ab397401 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:41.104316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:41.214445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346ab38-befb-446a-a5ad-b10101becdd2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:41.250774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:41.264604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:41.265444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:41.265732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:41.265980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:41.267174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] RBAC: Adding query filter params (role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=403d37853e1f4e059b26eb93ab397401, user.id=8573f95032444cceb9d70c7fdff22f68) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:41.267548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:41.269782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a343b2-1475-4560-9b34-a4f2dc794c02 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:41.278315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 566/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:41 2022] GET /identity/v3/role_assignments?user.id=8573f95032444cceb9d70c7fdff22f68&scope.project.id=403d37853e1f4e059b26eb93ab397401&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:43.224951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abca1e-578e-4553-aaef-5e1a3f14b47b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:43.225815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abca1e-578e-4553-aaef-5e1a3f14b47b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:43.225815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abca1e-578e-4553-aaef-5e1a3f14b47b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:43.228132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 552/1118] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:43.233649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:43.234058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:43.234185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:43.255054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:43.256202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:43.339196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 567/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:43.362682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abca1e-578e-4553-aaef-5e1a3f14b47b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:43.432942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:43.490471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abca1e-578e-4553-aaef-5e1a3f14b47b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:43.525388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:43.525948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:43.526222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:43.526490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] PATH_INFO: `/v3/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:43.528139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] RBAC: Authorizing `identity:get_role(role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:43.529502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:43.530819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 553/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:43 2022] GET /identity/v3/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:43.537203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:43.649700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36de548-e954-43b8-a415-184828c94bb4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:43.672952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:43.686715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:43.687348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:43.687627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:43.687938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] PATH_INFO: `/v3/users/8573f95032444cceb9d70c7fdff22f68` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:43.689125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] RBAC: Authorizing `identity:get_user(user_id=8573f95032444cceb9d70c7fdff22f68)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:43.690561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:43.692157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 568/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:43 2022] GET /identity/v3/users/8573f95032444cceb9d70c7fdff22f68 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:43.698140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:43.818666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eaf08a-49d1-4c0a-9a0c-d4bccb40c00b None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:43.844560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:43.856791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecbbd2-c540-4df5-8176-43e52fb95e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:43.857451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecbbd2-c540-4df5-8176-43e52fb95e46 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:43.857732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecbbd2-c540-4df5-8176-43e52fb95e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:43.858006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecbbd2-c540-4df5-8176-43e52fb95e46 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:43.859271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecbbd2-c540-4df5-8176-43e52fb95e46 None admin] RBAC: Authorizing `identity:get_project(project_id=403d37853e1f4e059b26eb93ab397401)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:43.860714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecbbd2-c540-4df5-8176-43e52fb95e46 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:43.862365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 554/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:43 2022] GET /identity/v3/projects/403d37853e1f4e059b26eb93ab397401 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:43.868408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:43.970174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7a926-d76f-494c-92bc-e8c7a049051a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:44.006238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736666e-1ec2-4281-953a-7d1384923715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:44.006911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736666e-1ec2-4281-953a-7d1384923715 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:44.007194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736666e-1ec2-4281-953a-7d1384923715 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:44.007474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736666e-1ec2-4281-953a-7d1384923715 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401/users/8573f95032444cceb9d70c7fdff22f68/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:44.010096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736666e-1ec2-4281-953a-7d1384923715 None admin] RBAC: Authorizing `identity:create_grant(project_id=403d37853e1f4e059b26eb93ab397401, user_id=8573f95032444cceb9d70c7fdff22f68, role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:44.011541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736666e-1ec2-4281-953a-7d1384923715 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:44.028358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 569/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:01:43 2022] PUT /identity/v3/projects/403d37853e1f4e059b26eb93ab397401/users/8573f95032444cceb9d70c7fdff22f68/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:01:45.977440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dbd4-7dcf-429e-a215-b4f6de7ae34a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:45.978057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dbd4-7dcf-429e-a215-b4f6de7ae34a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:45.978057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dbd4-7dcf-429e-a215-b4f6de7ae34a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:45.979391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 555/1124] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:45.988165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:45.988897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:45.989059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:46.016392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:46.017598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:46.065944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:46.088998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 570/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:46.112898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7dbd4-7dcf-429e-a215-b4f6de7ae34a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:46.222420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7dbd4-7dcf-429e-a215-b4f6de7ae34a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:46.256915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:46.257469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:46.257742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:46.258010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] PATH_INFO: `/v3/roles/b45b6545a9d3485a88cd9433637a1a81` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:46.259465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] RBAC: Authorizing `identity:get_role(role_id=b45b6545a9d3485a88cd9433637a1a81)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:46.260860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:46.262176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 556/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:46 2022] GET /identity/v3/roles/b45b6545a9d3485a88cd9433637a1a81 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:46.271310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:46.378400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0740d2-192d-42b9-9494-4b0aebc622d8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:46.412898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:46.413522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:46.413802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:46.414071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] PATH_INFO: `/v3/users/8573f95032444cceb9d70c7fdff22f68` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:46.415317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] RBAC: Authorizing `identity:get_user(user_id=8573f95032444cceb9d70c7fdff22f68)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:46.416791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:46.418343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 571/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:46 2022] GET /identity/v3/users/8573f95032444cceb9d70c7fdff22f68 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:46.427199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:46.487901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:46.530682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf9987-c4c9-4065-ae30-20187e87ffcd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:46.567613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:46.568247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:46.568528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:46.568801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:46.646861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:46.648339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:46.706609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:46.752150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 557/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:01:46.758224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:46.809591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:46.861802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99101ec-583f-4c1a-a9ef-deb82834f85f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:46.897115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2fc71-139f-4491-a37e-34ab0c034590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:46.897739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2fc71-139f-4491-a37e-34ab0c034590 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:46.898020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2fc71-139f-4491-a37e-34ab0c034590 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:46.898290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2fc71-139f-4491-a37e-34ab0c034590 None admin] PATH_INFO: `/v3/projects/403d37853e1f4e059b26eb93ab397401` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:46.899487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2fc71-139f-4491-a37e-34ab0c034590 None admin] RBAC: Authorizing `identity:get_project(project_id=403d37853e1f4e059b26eb93ab397401)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:46.900960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2fc71-139f-4491-a37e-34ab0c034590 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:46.902603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 572/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:46 2022] GET /identity/v3/projects/403d37853e1f4e059b26eb93ab397401 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:46.909800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:47.025339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc720-2007-465d-952f-7eadf430e142 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:47.063393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:47.063976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:47.064265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:47.064542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:47.065332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] RBAC: Adding query filter params (role.id=b45b6545a9d3485a88cd9433637a1a81, scope.project.id=403d37853e1f4e059b26eb93ab397401, user.id=8573f95032444cceb9d70c7fdff22f68) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:47.065648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:47.067091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b6a55-b618-4167-b79a-ee61ed753441 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:47.073243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 558/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:46 2022] GET /identity/v3/role_assignments?user.id=8573f95032444cceb9d70c7fdff22f68&scope.project.id=403d37853e1f4e059b26eb93ab397401&role.id=b45b6545a9d3485a88cd9433637a1a81 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:49.061870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee30c4e-0599-4987-84cb-ac01e974e6d5 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:49.062728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee30c4e-0599-4987-84cb-ac01e974e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:49.062728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee30c4e-0599-4987-84cb-ac01e974e6d5 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:49.065007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 573/1131] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:49.070418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3930-4134-490d-b842-6be321824b78 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:49.070856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3930-4134-490d-b842-6be321824b78 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:49.070947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3930-4134-490d-b842-6be321824b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:49.089639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4eab3930-4134-490d-b842-6be321824b78 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:49.090814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4eab3930-4134-490d-b842-6be321824b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:49.108379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:49.160014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 559/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:49.184133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee30c4e-0599-4987-84cb-ac01e974e6d5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:49.304908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee30c4e-0599-4987-84cb-ac01e974e6d5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:49.327116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:49.347384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4b159-efe0-4a1d-ae8e-91ad164015c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:49.348051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4b159-efe0-4a1d-ae8e-91ad164015c4 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:49.348337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4b159-efe0-4a1d-ae8e-91ad164015c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:49.348613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4b159-efe0-4a1d-ae8e-91ad164015c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:49.349921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4b159-efe0-4a1d-ae8e-91ad164015c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:49.351375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4b159-efe0-4a1d-ae8e-91ad164015c4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:49.353021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 574/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:49.360522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3930-4134-490d-b842-6be321824b78 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:49.463492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3930-4134-490d-b842-6be321824b78 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:49.502433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362ff69-6df8-4a8a-9936-b95e6e3ee80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:49.503045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362ff69-6df8-4a8a-9936-b95e6e3ee80f None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:49.503323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362ff69-6df8-4a8a-9936-b95e6e3ee80f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:49.503573 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362ff69-6df8-4a8a-9936-b95e6e3ee80f None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:49.504290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362ff69-6df8-4a8a-9936-b95e6e3ee80f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:49.505822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362ff69-6df8-4a8a-9936-b95e6e3ee80f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:49.536475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 560/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:01:49 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:51.513569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7ccf-6b99-4b8f-be3a-5606d70ef336 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:51.514453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7ccf-6b99-4b8f-be3a-5606d70ef336 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:51.514453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7ccf-6b99-4b8f-be3a-5606d70ef336 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:51.516671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 575/1135] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:51.522879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:51.523596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:51.523803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:51.544376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:51.550051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:51.551966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:51.630653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 561/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:51.656079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7ccf-6b99-4b8f-be3a-5606d70ef336 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:51.776975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7ccf-6b99-4b8f-be3a-5606d70ef336 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:51.810079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abaa132-84b5-4f66-8e18-f07eac666e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:51.810713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaa132-84b5-4f66-8e18-f07eac666e75 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:51.810992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaa132-84b5-4f66-8e18-f07eac666e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:51.811264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaa132-84b5-4f66-8e18-f07eac666e75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:51.812461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaa132-84b5-4f66-8e18-f07eac666e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:51.813895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaa132-84b5-4f66-8e18-f07eac666e75 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:51.815475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 576/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:01:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:51.824716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:51.939888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11b94d-7e6c-41d3-a443-6a706f40158b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:51.974841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367aef3-6bb0-4269-9722-d9f788002bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:51.975464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367aef3-6bb0-4269-9722-d9f788002bc6 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:51.975745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367aef3-6bb0-4269-9722-d9f788002bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:51.976075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367aef3-6bb0-4269-9722-d9f788002bc6 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:51.976749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367aef3-6bb0-4269-9722-d9f788002bc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:51.978222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367aef3-6bb0-4269-9722-d9f788002bc6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:51.987082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:52.003433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 562/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:01:51 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:01:53.996712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837c37f-ba95-47f7-9f90-d7b94322d16f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:53.997291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837c37f-ba95-47f7-9f90-d7b94322d16f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:53.997291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837c37f-ba95-47f7-9f90-d7b94322d16f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:53.998624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 577/1139] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:54.006527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:54.006955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:54.007053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:54.025867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:54.027039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:54.096575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 563/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:54.120358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837c37f-ba95-47f7-9f90-d7b94322d16f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:54.179139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:54.232871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837c37f-ba95-47f7-9f90-d7b94322d16f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:54.270767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:54.271337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:54.271611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:54.271910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:54.282150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:54.287026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:54.288194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:54.289195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 578/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:54.294503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:54.395691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4834-17bd-4359-b992-63f47fc32135 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:54.417978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:54.431343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:54.431981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:54.432259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:54.432530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:54.433242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:54.433544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:54.434934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:54.442730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 564/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:54.448647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:54.553569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e522ce4-1147-491c-9e40-4335f2e7394f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:54.566629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:54.589169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:54.589786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:54.590066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:54.590341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] PATH_INFO: `/v3/users/e7cd0280f36249298f9497f6b9b57248` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:54.603485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] RBAC: Authorizing `identity:get_user(user_id=e7cd0280f36249298f9497f6b9b57248)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:54.604952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:54.606623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 579/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:54 2022] GET /identity/v3/users/e7cd0280f36249298f9497f6b9b57248 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:54.616232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:54.725321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257a3cb-f0e0-425c-acf6-5b91bd6e59f9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:54.759735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:54.760367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:54.760648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:54.760914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:54.811857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:54.832006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:54.833462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:54.933307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 565/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:01:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:01:54.940178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:55.051464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35023a0b-fffc-488a-ae31-a911bab0787c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:55.101348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b205cf3-7aeb-468d-8a87-b8e948748272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:55.101929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205cf3-7aeb-468d-8a87-b8e948748272 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:55.102238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205cf3-7aeb-468d-8a87-b8e948748272 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:55.102513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205cf3-7aeb-468d-8a87-b8e948748272 None admin] PATH_INFO: `/v3/projects/00b35791c1f245fca13e5bee3f97e772` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:55.103730 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b205cf3-7aeb-468d-8a87-b8e948748272 None admin] RBAC: Authorizing `identity:get_project(project_id=00b35791c1f245fca13e5bee3f97e772)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:55.105219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b205cf3-7aeb-468d-8a87-b8e948748272 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:55.106893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 580/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:54 2022] GET /identity/v3/projects/00b35791c1f245fca13e5bee3f97e772 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:01:55.113311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:55.197832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:55.213547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251013-9117-405d-aef1-1b85dcdd86c3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:55.248659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:55.249267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:55.249541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:55.249811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:55.250596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=00b35791c1f245fca13e5bee3f97e772, user.id=e7cd0280f36249298f9497f6b9b57248) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:55.250904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:55.252368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b01f04-1efb-4b5f-affb-a8754da4fdf0 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:55.257975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 566/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:01:55 2022] GET /identity/v3/role_assignments?user.id=e7cd0280f36249298f9497f6b9b57248&scope.project.id=00b35791c1f245fca13e5bee3f97e772&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:57.205839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d880c-c3c5-4b78-b677-a2bcda2eef2d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:57.206735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d880c-c3c5-4b78-b677-a2bcda2eef2d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:57.206735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d880c-c3c5-4b78-b677-a2bcda2eef2d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:57.208950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 581/1147] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:01:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:01:57.214403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:57.214822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:57.214939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:57.233186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:01:57.234419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:01:57.304404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 567/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:01:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:01:57.326490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d880c-c3c5-4b78-b677-a2bcda2eef2d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:57.371915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:57.447107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d880c-c3c5-4b78-b677-a2bcda2eef2d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:57.486975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:57.487594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:57.487905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:57.488175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:57.498988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:01:57.503800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:01:57.505024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:01:57.506095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 582/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:01:57.510925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:57.590711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:57.616564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0077d4-cc70-4417-aede-504077dc1b7b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:57.653154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:57.653758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:57.654040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:57.654313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:57.655023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:01:57.655327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:57.656742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:57.665505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 568/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:57.671807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:57.766720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:01:57.770737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c9762-adf0-4a17-9bee-2277af3db5b8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:57.805570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:57.806191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:57.806467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:57.806737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] PATH_INFO: `/v3/users/e7cd0280f36249298f9497f6b9b57248` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:57.807960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] RBAC: Authorizing `identity:get_user(user_id=e7cd0280f36249298f9497f6b9b57248)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:57.809384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:57.810957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 583/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:01:57 2022] GET /identity/v3/users/e7cd0280f36249298f9497f6b9b57248 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:57.816910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:57.923432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c225ba-1b02-4b14-b3bb-31a6fbda33de None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:57.958537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebe95a-b720-46e6-9da5-9798ea631fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:57.959163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe95a-b720-46e6-9da5-9798ea631fdd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:57.959440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe95a-b720-46e6-9da5-9798ea631fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:57.959710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebe95a-b720-46e6-9da5-9798ea631fdd None admin] PATH_INFO: `/v3/projects/00b35791c1f245fca13e5bee3f97e772` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:57.960943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebe95a-b720-46e6-9da5-9798ea631fdd None admin] RBAC: Authorizing `identity:get_project(project_id=00b35791c1f245fca13e5bee3f97e772)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:57.962356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebe95a-b720-46e6-9da5-9798ea631fdd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:57.964000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 569/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:01:57 2022] GET /identity/v3/projects/00b35791c1f245fca13e5bee3f97e772 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:01:57.971643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:01:58.075367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb92d5-50d6-4ace-a9d4-52fd2d76c81f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:01:58.114627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c2c1c-e5e8-4215-b4a7-4bf6602a4345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:01:58.115281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c2c1c-e5e8-4215-b4a7-4bf6602a4345 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:01:58.115562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c2c1c-e5e8-4215-b4a7-4bf6602a4345 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:01:58.115846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c2c1c-e5e8-4215-b4a7-4bf6602a4345 None admin] PATH_INFO: `/v3/projects/00b35791c1f245fca13e5bee3f97e772/users/e7cd0280f36249298f9497f6b9b57248/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:01:58.117880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c2c1c-e5e8-4215-b4a7-4bf6602a4345 None admin] RBAC: Authorizing `identity:create_grant(project_id=00b35791c1f245fca13e5bee3f97e772, user_id=e7cd0280f36249298f9497f6b9b57248, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:01:58.119307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c2c1c-e5e8-4215-b4a7-4bf6602a4345 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:01:58.134928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 584/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:01:57 2022] PUT /identity/v3/projects/00b35791c1f245fca13e5bee3f97e772/users/e7cd0280f36249298f9497f6b9b57248/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:02:00.094725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abe1c2-e8e0-4400-88cd-b3b544261b5e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:00.095625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abe1c2-e8e0-4400-88cd-b3b544261b5e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:00.095625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abe1c2-e8e0-4400-88cd-b3b544261b5e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:00.097811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 570/1154] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:00.103121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:00.103533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:00.103622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:00.123280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:00.124498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:00.144680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:00.192843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 585/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:00.215740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abe1c2-e8e0-4400-88cd-b3b544261b5e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:00.228605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:00.324995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abe1c2-e8e0-4400-88cd-b3b544261b5e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:00.358709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:00.359387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:00.359663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:00.359960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:00.370689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:02:00.375526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:02:00.376623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:02:00.377779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 571/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:02:00.382866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:00.490022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71996f27-3091-45c6-9e7a-e5c38ca7d3c3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:00.528169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:00.528779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:00.529059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:00.529329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:00.530046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:00.530353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:00.531754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:00.539745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 586/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:02:00.548396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:00.619531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:00.653280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03f531-30b2-4a44-b4c6-ad8389e0fd42 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:00.688054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:00.688682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:00.688962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:00.689239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] PATH_INFO: `/v3/users/e7cd0280f36249298f9497f6b9b57248` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:00.690518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] RBAC: Authorizing `identity:get_user(user_id=e7cd0280f36249298f9497f6b9b57248)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:00.691980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:00.693597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 572/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:02:00 2022] GET /identity/v3/users/e7cd0280f36249298f9497f6b9b57248 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:00.700253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:00.722562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:00.828145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81aba4-cee0-4383-bd9d-ad1c8059d9f2 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:00.866276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:00.866891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:00.867177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:00.867448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:00.935802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:00.937315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:00.954903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:01.039173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 587/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:02:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:02:01.045545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:01.152943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b4cf5-8f40-40c8-8bd4-080292188e24 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:01.187251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8560f-8270-4b6d-b9a9-ea07814092aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:01.187810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8560f-8270-4b6d-b9a9-ea07814092aa None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:01.188169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8560f-8270-4b6d-b9a9-ea07814092aa None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:01.188441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8560f-8270-4b6d-b9a9-ea07814092aa None admin] PATH_INFO: `/v3/projects/00b35791c1f245fca13e5bee3f97e772` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:01.189731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8560f-8270-4b6d-b9a9-ea07814092aa None admin] RBAC: Authorizing `identity:get_project(project_id=00b35791c1f245fca13e5bee3f97e772)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:01.191168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8560f-8270-4b6d-b9a9-ea07814092aa None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:01.192891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 573/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:02:01 2022] GET /identity/v3/projects/00b35791c1f245fca13e5bee3f97e772 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:01.199578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:01.306204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306f7b-8ca7-4c0e-9383-2ecb9c661324 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:01.323340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:01.340405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:01.341014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:01.341293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:01.341558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:01.342367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=00b35791c1f245fca13e5bee3f97e772, user.id=e7cd0280f36249298f9497f6b9b57248) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:01.342670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:01.344137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88203bf-378c-4da2-ba38-89b6d36ee407 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:01.349763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 588/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:02:01 2022] GET /identity/v3/role_assignments?user.id=e7cd0280f36249298f9497f6b9b57248&scope.project.id=00b35791c1f245fca13e5bee3f97e772&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:03.346266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5522d-6537-4e3e-8390-f7e6226419a7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:03.347137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5522d-6537-4e3e-8390-f7e6226419a7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:03.347137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5522d-6537-4e3e-8390-f7e6226419a7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:03.349204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 574/1162] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:03.355824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:03.356526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:03.356709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:03.376559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:03.377743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:03.449186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 589/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:03.472775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5522d-6537-4e3e-8390-f7e6226419a7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:03.476939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:03.586548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5522d-6537-4e3e-8390-f7e6226419a7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:03.621184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:03.621729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:03.622014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:03.622288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:03.690476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:03.691951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:03.694519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:03.796853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 575/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:02:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:02:03.806507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:03.906717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eae9bf-8075-4837-b7d0-9a2ece71da3f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:03.919522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:03.942295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:03.942925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:03.943204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:03.943471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:03.950535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:03.952000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-dcd3891d-9b38-4f96-881c-75b691b0edbb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:02:03.964015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:02:03.965121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:02:03.966041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 590/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:02:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:02:03.971548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:04.074437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ce5a-a273-4863-91ff-281840adf709 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:04.076817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:04.109431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:04.110045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:04.110326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:04.110602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:04.111291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:04.111594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:04.113001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42ae2-349b-4c41-a3dd-d0b9958017d1 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:04.120179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 576/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Fri Aug 5 05:02:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:06.063323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95542f-7dec-4d82-96e7-48c7ce7f4e89 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:06.063924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95542f-7dec-4d82-96e7-48c7ce7f4e89 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:06.063924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95542f-7dec-4d82-96e7-48c7ce7f4e89 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:06.065263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 591/1167] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:06.071550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7a4a6-6753-45a2-95b4-dcb94319a3ac None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:06.072354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7a4a6-6753-45a2-95b4-dcb94319a3ac None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:06.072474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7a4a6-6753-45a2-95b4-dcb94319a3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:06.099979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b0d7a4a6-6753-45a2-95b4-dcb94319a3ac None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:06.101169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b0d7a4a6-6753-45a2-95b4-dcb94319a3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:06.168647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 577/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:06.199277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95542f-7dec-4d82-96e7-48c7ce7f4e89 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:06.316994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95542f-7dec-4d82-96e7-48c7ce7f4e89 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:06.350568 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af534c19-4ec6-4263-b6e8-4c03900f3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:06.351199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534c19-4ec6-4263-b6e8-4c03900f3654 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:06.351477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534c19-4ec6-4263-b6e8-4c03900f3654 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:06.351749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af534c19-4ec6-4263-b6e8-4c03900f3654 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:06.352489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af534c19-4ec6-4263-b6e8-4c03900f3654 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:06.353880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af534c19-4ec6-4263-b6e8-4c03900f3654 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:06.375210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 592/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Fri Aug 5 05:02:06 2022] POST /identity/v3/domains => generated 267 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:02:08.356667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a1fa-4d67-45cf-be45-edb881200ecd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:08.357491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a1fa-4d67-45cf-be45-edb881200ecd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:08.357491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a1fa-4d67-45cf-be45-edb881200ecd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:08.359369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 578/1170] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:08.368069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576085-d998-4583-9cc3-67119840722c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:08.368482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576085-d998-4583-9cc3-67119840722c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:08.368690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576085-d998-4583-9cc3-67119840722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:08.389972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b5576085-d998-4583-9cc3-67119840722c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:08.391159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5576085-d998-4583-9cc3-67119840722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:08.418152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:08.461330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 593/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:08.485058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a1fa-4d67-45cf-be45-edb881200ecd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:08.594648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:08.599720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a1fa-4d67-45cf-be45-edb881200ecd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:08.635722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cc53a-333d-4d34-95ef-96643b3d925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:08.636327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc53a-333d-4d34-95ef-96643b3d925d None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:08.636602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc53a-333d-4d34-95ef-96643b3d925d None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:08.636871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc53a-333d-4d34-95ef-96643b3d925d None admin] PATH_INFO: `/v3/domains/b9a130fd8fd140cf8abe813c412c00a5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:08.647707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cc53a-333d-4d34-95ef-96643b3d925d None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9a130fd8fd140cf8abe813c412c00a5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:08.649182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cc53a-333d-4d34-95ef-96643b3d925d None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:08.650756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 579/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Fri Aug 5 05:02:08 2022] GET /identity/v3/domains/b9a130fd8fd140cf8abe813c412c00a5 => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:08.656719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5576085-d998-4583-9cc3-67119840722c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:08.767809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5576085-d998-4583-9cc3-67119840722c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:08.802970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0b4e5-ee83-4d0d-b6c0-5057e84b03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:08.803574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0b4e5-ee83-4d0d-b6c0-5057e84b03b2 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:08.803881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0b4e5-ee83-4d0d-b6c0-5057e84b03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:08.804153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0b4e5-ee83-4d0d-b6c0-5057e84b03b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:08.804829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0b4e5-ee83-4d0d-b6c0-5057e84b03b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:08.806258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0b4e5-ee83-4d0d-b6c0-5057e84b03b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:08.835396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 594/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:02:08 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:02:10.863486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c9c7c-0057-4a29-990b-c6449c385dcd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:10.864085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c9c7c-0057-4a29-990b-c6449c385dcd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:10.864085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c9c7c-0057-4a29-990b-c6449c385dcd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:10.866144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 580/1174] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:10.874732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:10.875132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:10.875342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:10.899671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:10.900885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:10.910286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:10.980275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 595/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:11.004949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c9c7c-0057-4a29-990b-c6449c385dcd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:11.114473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c9c7c-0057-4a29-990b-c6449c385dcd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:11.147990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3af44-17b9-4b7b-affb-63b280c047cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:11.148617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3af44-17b9-4b7b-affb-63b280c047cc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:11.148893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3af44-17b9-4b7b-affb-63b280c047cc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:11.149146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3af44-17b9-4b7b-affb-63b280c047cc None admin] PATH_INFO: `/v3/domains/b9a130fd8fd140cf8abe813c412c00a5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:11.150311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3af44-17b9-4b7b-affb-63b280c047cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9a130fd8fd140cf8abe813c412c00a5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:11.151744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3af44-17b9-4b7b-affb-63b280c047cc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:11.153377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 581/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Fri Aug 5 05:02:11 2022] GET /identity/v3/domains/b9a130fd8fd140cf8abe813c412c00a5 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:11.162076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:11.265653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683d338-243b-412c-a6e0-076fa8945790 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:11.303574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daa6c5-53f4-4889-a034-4095e7ec67bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:11.304220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daa6c5-53f4-4889-a034-4095e7ec67bd None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:11.304499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daa6c5-53f4-4889-a034-4095e7ec67bd None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:11.304776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daa6c5-53f4-4889-a034-4095e7ec67bd None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:11.305463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0daa6c5-53f4-4889-a034-4095e7ec67bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:11.306907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0daa6c5-53f4-4889-a034-4095e7ec67bd None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:11.326803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:11.332400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 596/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:02:11 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:02:13.329685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cc709-4762-4bd2-bd2e-e7e66af10e11 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:13.330546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cc709-4762-4bd2-bd2e-e7e66af10e11 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:13.330546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cc709-4762-4bd2-bd2e-e7e66af10e11 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:13.332714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 582/1178] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:13.341566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:13.341968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:13.342186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:13.367123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:13.368349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:13.450717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 597/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:13.474794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cc709-4762-4bd2-bd2e-e7e66af10e11 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:13.514464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:13.591531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cc709-4762-4bd2-bd2e-e7e66af10e11 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:13.625327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:13.625962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:13.626239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:13.626514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:13.637632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:02:13.642536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:02:13.643682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:02:13.645252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 583/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:02:13.656280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:13.758648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f5167-d207-4f4b-aa34-7514adf3d497 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:13.788031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:13.794675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:13.795285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:13.795578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:13.795854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:13.796586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:13.796885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:13.798257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:13.806656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 598/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:02:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:13.812858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:13.903210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:13.920426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3654b0-94bd-471e-8789-61c4f2f61a91 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:13.957261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:13.957887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:13.958170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:13.958442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] PATH_INFO: `/v3/users/969ccba23e4f4442a62e46d09a5af57f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:13.971883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] RBAC: Authorizing `identity:get_user(user_id=969ccba23e4f4442a62e46d09a5af57f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:13.973321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:13.974910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 584/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:02:13 2022] GET /identity/v3/users/969ccba23e4f4442a62e46d09a5af57f => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:13.980806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:14.081079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169216-43a9-423f-9116-a2b442c44085 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:14.114928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:14.115531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:14.115805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:14.116101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:14.174064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:14.189506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:14.190981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:14.288107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 599/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:02:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:02:14.295216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:14.408522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f974bb-559d-4150-bb6c-8f36222c783c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:14.445859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088a954-1851-4e7b-bd19-7affae721136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:14.446421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088a954-1851-4e7b-bd19-7affae721136 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:14.446697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088a954-1851-4e7b-bd19-7affae721136 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:14.446969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088a954-1851-4e7b-bd19-7affae721136 None admin] PATH_INFO: `/v3/projects/f0e0de6ed6d040638a76051335db8b8a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:14.448189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088a954-1851-4e7b-bd19-7affae721136 None admin] RBAC: Authorizing `identity:get_project(project_id=f0e0de6ed6d040638a76051335db8b8a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:14.449624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088a954-1851-4e7b-bd19-7affae721136 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:14.451264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 585/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:02:14 2022] GET /identity/v3/projects/f0e0de6ed6d040638a76051335db8b8a => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:14.458153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:14.558868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:14.569070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d82b33-cb5c-4a3d-8145-2d507feaa584 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:14.604328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:14.604931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:14.605209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:14.605478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:14.606256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=f0e0de6ed6d040638a76051335db8b8a, user.id=969ccba23e4f4442a62e46d09a5af57f) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:14.606561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:14.608038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52652d-3978-478d-b4d9-d3c21de1267e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:14.613667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 600/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:02:14 2022] GET /identity/v3/role_assignments?user.id=969ccba23e4f4442a62e46d09a5af57f&scope.project.id=f0e0de6ed6d040638a76051335db8b8a&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:16.562915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8685c-0525-4069-84aa-696c044b289e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:16.563532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8685c-0525-4069-84aa-696c044b289e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:16.563532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8685c-0525-4069-84aa-696c044b289e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:16.564844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 586/1186] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:16.571137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:16.571939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:16.572106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:16.593371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:16.594593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:16.662891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 601/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:16.685701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8685c-0525-4069-84aa-696c044b289e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:16.702665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:16.792678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8685c-0525-4069-84aa-696c044b289e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:16.829034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:16.829587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:16.829864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:16.830131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:16.840738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:02:16.845585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:02:16.846725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:02:16.847957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 587/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:02:16.852855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:16.948678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:16.964845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83404855-fc59-4b74-b88e-1212ab1b1ed9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:17.001555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:17.002186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:17.002469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:17.002742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:17.003459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:17.003785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:17.005169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:17.013665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 602/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:17.022403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:17.100090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:17.142041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532f771-91eb-4ba6-b799-e556982b1b09 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:17.176473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:17.177103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:17.177382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:17.177653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] PATH_INFO: `/v3/users/969ccba23e4f4442a62e46d09a5af57f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:17.178859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] RBAC: Authorizing `identity:get_user(user_id=969ccba23e4f4442a62e46d09a5af57f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:17.180319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:17.181909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 588/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:02:17 2022] GET /identity/v3/users/969ccba23e4f4442a62e46d09a5af57f => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:17.190606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:17.298192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91e86-2721-4554-afc7-644056c97aea None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:17.332763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25f28f-cb62-4bd1-93dc-44f8ea2708ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:17.333384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25f28f-cb62-4bd1-93dc-44f8ea2708ef None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:17.333658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25f28f-cb62-4bd1-93dc-44f8ea2708ef None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:17.333928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25f28f-cb62-4bd1-93dc-44f8ea2708ef None admin] PATH_INFO: `/v3/projects/f0e0de6ed6d040638a76051335db8b8a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:17.335198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25f28f-cb62-4bd1-93dc-44f8ea2708ef None admin] RBAC: Authorizing `identity:get_project(project_id=f0e0de6ed6d040638a76051335db8b8a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:17.336655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25f28f-cb62-4bd1-93dc-44f8ea2708ef None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:17.338294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 603/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:02:17 2022] GET /identity/v3/projects/f0e0de6ed6d040638a76051335db8b8a => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:02:17.345020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:17.452760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2c501-b80a-4b48-8709-c332a6671e0c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:17.488505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1baa-7fa4-4cc7-8d59-c8325d2e2976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:17.489166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1baa-7fa4-4cc7-8d59-c8325d2e2976 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:17.489443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1baa-7fa4-4cc7-8d59-c8325d2e2976 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:17.489714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1baa-7fa4-4cc7-8d59-c8325d2e2976 None admin] PATH_INFO: `/v3/projects/f0e0de6ed6d040638a76051335db8b8a/users/969ccba23e4f4442a62e46d09a5af57f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:17.491912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1baa-7fa4-4cc7-8d59-c8325d2e2976 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0e0de6ed6d040638a76051335db8b8a, user_id=969ccba23e4f4442a62e46d09a5af57f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:17.493320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1baa-7fa4-4cc7-8d59-c8325d2e2976 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:17.504881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:17.510453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 589/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:02:17 2022] PUT /identity/v3/projects/f0e0de6ed6d040638a76051335db8b8a/users/969ccba23e4f4442a62e46d09a5af57f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:02:19.447157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354ba1-b1f6-43c1-8112-cb20be028e0f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:19.447732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354ba1-b1f6-43c1-8112-cb20be028e0f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:19.447732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354ba1-b1f6-43c1-8112-cb20be028e0f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:19.449087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 604/1193] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:02:19.454931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:19.455374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:19.455473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:19.476040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:02:19.477576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:02:19.551782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 590/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:02:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:02:19.575226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd354ba1-b1f6-43c1-8112-cb20be028e0f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:19.602795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:19.704053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd354ba1-b1f6-43c1-8112-cb20be028e0f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:19.741278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:19.741910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:19.742189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:19.742460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:19.753138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:02:19.757981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:02:19.759119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:02:19.760341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 605/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:02:19.765257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:19.865253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe6bb2-caf5-4c5d-8f7e-e61b83bd50e0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:19.894134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:19.901335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:19.901959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:19.902225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:19.902498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:19.903213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:19.903514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:19.904915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:19.913131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 591/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:02:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:19.918888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:19.999014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:20.022652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eb1ca-a67e-4055-a471-651f8950de08 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:20.057461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:20.058094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:20.058369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:20.058640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] PATH_INFO: `/v3/users/969ccba23e4f4442a62e46d09a5af57f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:20.059871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=969ccba23e4f4442a62e46d09a5af57f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:20.061299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:20.062895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 606/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Fri Aug 5 05:02:19 2022] GET /identity/v3/users/969ccba23e4f4442a62e46d09a5af57f => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:20.072565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:20.184836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f3644-6075-4a53-8e6e-367fba5a8af9 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:20.221040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:20.221647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:20.221925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:20.222212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:20.283742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:20.299273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:20.301193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:20.402391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 592/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:02:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:02:20.413443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:20.529538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b64e8-c8af-4999-ad3c-f7943ae7b7c0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:20.565049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166c722-16c8-467a-8901-fc2bd1eda23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:20.565616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166c722-16c8-467a-8901-fc2bd1eda23e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:20.565896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166c722-16c8-467a-8901-fc2bd1eda23e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:20.566167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166c722-16c8-467a-8901-fc2bd1eda23e None admin] PATH_INFO: `/v3/projects/f0e0de6ed6d040638a76051335db8b8a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:20.567447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166c722-16c8-467a-8901-fc2bd1eda23e None admin] RBAC: Authorizing `identity:get_project(project_id=f0e0de6ed6d040638a76051335db8b8a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:20.568912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166c722-16c8-467a-8901-fc2bd1eda23e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:20.570584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 607/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Fri Aug 5 05:02:20 2022] GET /identity/v3/projects/f0e0de6ed6d040638a76051335db8b8a => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:02:20.577049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:02:20.674074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:20.683649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35aba-38a0-4b47-bf6f-49fac84858d2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:02:20.716133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:02:20.716748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:02:20.717022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:02:20.717292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:02:20.718068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=f0e0de6ed6d040638a76051335db8b8a, user.id=969ccba23e4f4442a62e46d09a5af57f) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:02:20.718372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:02:20.719870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b9816-ed60-4c0c-a7ba-ae6ce8423c03 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:02:20.725690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 593/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:02:20 2022] GET /identity/v3/role_assignments?user.id=969ccba23e4f4442a62e46d09a5af57f&scope.project.id=f0e0de6ed6d040638a76051335db8b8a&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:02:30.559830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:02:30.723087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=22 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:15.578714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cba80-78d9-45b2-903a-ec00c3baa43d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:15.579376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cba80-78d9-45b2-903a-ec00c3baa43d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:15.579376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cba80-78d9-45b2-903a-ec00c3baa43d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:15.582337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 608/1201] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:15.586451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:15.586918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:15.587028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:15.609471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:15.610803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:15.699799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 594/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:15.724906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cba80-78d9-45b2-903a-ec00c3baa43d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:15.870886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cba80-78d9-45b2-903a-ec00c3baa43d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:15.929753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:15.930355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:15.930662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:15.930919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:15.942538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:15.944027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:15.960927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:15.962175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:15.963562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:15.963562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:15.963562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:15.963562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:15.963562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 609/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:15.968624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:16.115175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6b896-f5b5-4c25-a948-ef3ec6d18f05 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:16.167651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:16.174883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:16.184542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:16.184901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:16.185337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:16.186442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:16.186878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:16.191182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647b0e9-cc53-4642-96be-aed5e79dbabc None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:16.209550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 595/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:16.215014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:16.277774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:16.332668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c0fa2-0516-4de4-b542-179dfa8fbfce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:16.370846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2977a0a-6087-4737-a0d3-bfd1bd1e9e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:16.371451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977a0a-6087-4737-a0d3-bfd1bd1e9e12 None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:16.371755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977a0a-6087-4737-a0d3-bfd1bd1e9e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:16.372081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977a0a-6087-4737-a0d3-bfd1bd1e9e12 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:16.372854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2977a0a-6087-4737-a0d3-bfd1bd1e9e12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:16.374640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2977a0a-6087-4737-a0d3-bfd1bd1e9e12 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:16.403819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 610/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Fri Aug 5 05:03:16 2022] POST /identity/v3/users => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:03:18.606401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea57134-9881-4326-8f5d-4b62ead892af None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:18.607030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea57134-9881-4326-8f5d-4b62ead892af None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:18.607030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea57134-9881-4326-8f5d-4b62ead892af None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:18.608548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 596/1206] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:03:18.614568 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:18.615072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:18.615281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:18.635989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:18.637252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:18.725884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 611/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:18.749843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea57134-9881-4326-8f5d-4b62ead892af None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:18.874440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea57134-9881-4326-8f5d-4b62ead892af None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:18.919124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:18.919793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:18.920094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:18.920353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:18.932614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:03:18.937684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:18.938783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:18.939680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 597/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:18.946678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:18.992092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:19.069705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb462949-1809-4f74-9852-77eeb4baca22 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:19.107909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:19.108635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:19.108919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:19.109169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:19.110157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:19.110496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:19.112257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:19.126119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 612/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:03:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:19.132426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:19.192588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:19.252040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44b054-d63d-4b5f-8b06-a85a4ba5ea94 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:19.293766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:19.294504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:19.294786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:19.295063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:19.307102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:19.308779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:19.325234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:19.326281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:19.327383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:19.327383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:19.327383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:19.327383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:19.327383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 598/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:19.332535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:19.452658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:19.455668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c9f95-967d-4137-ad48-772a6835a72c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:19.493661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:19.494495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:19.494495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:19.494830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:19.495457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:19.495769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:19.497168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:19.509978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 613/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:03:19.516105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:19.635194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679c80f-c3a8-4a94-9a2b-18fa3b7ecd9c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:19.636759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:19.673020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:19.673671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:19.673986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:19.674254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:19.682964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:19.684433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:03:19.697569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:19.698804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:19.699810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 599/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:19.710484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:19.825250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a1d31-18b5-4127-ba38-0dab51dd3b97 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:19.860910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:19.861535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:19.861817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:19.862089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:19.862872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:19.863175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:19.864639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:19.883696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 614/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Fri Aug 5 05:03:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:19.889430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:20.008322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913b8a-0295-4148-8807-fe03e3be010c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:20.043957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:20.044593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:20.044874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:20.045186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:20.084434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:20.124648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:20.128372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:20.230276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 600/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:03:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 05:03:20.236358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:20.259234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:20.349536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113a733-e845-4692-b421-5595cecc40ec None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:20.389679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:20.390404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:20.390739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:20.391113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:20.402180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:20.403737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:20.419661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:20.420866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:20.422434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:20.422434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:20.422434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:20.422434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:20.422434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 615/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:20.427636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:20.505846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:20.548625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edfa20-fe57-4cec-9e8d-a6d5eb73a0f5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:20.590724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:20.591646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:20.591955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:20.592313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:20.593028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:20.593395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:20.595043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:20.607830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 601/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:20.613535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:20.690146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:20.724887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25ece-0dee-4447-be04-e1ec3347a728 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:20.769412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:20.770128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:20.770417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:20.770689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:20.779630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:20.781675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:03:20.794889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:20.796167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:20.797141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 616/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:03:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:20.802648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:20.919549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754ca7f-61c3-45a2-b785-a3349c65bad5 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:20.942152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:20.954903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:20.955682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:20.956047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:20.956364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:20.957275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:20.957624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:20.959245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69793d85-3203-481d-8f46-3b27e9ebbe87 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:20.971467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 602/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:03:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:03:20.977618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:21.097443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1073737-0b29-4e1b-a438-0cd74bdd8dd5 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:21.131470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:21.140479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:21.141315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:21.141597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:21.141868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:21.142664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=7b3c6d4d4371427ca4ca74a962a47967) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:21.142974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:21.144526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdfe8a-5a51-4adf-bd35-6be07696e2b2 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:21.153324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 617/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:03:20 2022] GET /identity/v3/role_assignments?user.id=7b3c6d4d4371427ca4ca74a962a47967&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:23.224610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479944bf-ecff-4c1f-9db1-c2c756127d7e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:23.225254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479944bf-ecff-4c1f-9db1-c2c756127d7e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:23.225254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479944bf-ecff-4c1f-9db1-c2c756127d7e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:23.226849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 603/1220] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:03:23.232201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:23.232650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:23.232749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:23.253373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:23.254625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:23.330562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 618/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:03:23.352327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479944bf-ecff-4c1f-9db1-c2c756127d7e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:23.461390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479944bf-ecff-4c1f-9db1-c2c756127d7e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:23.503845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:23.504238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:23.504514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:23.504789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:23.518285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:03:23.524199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:23.525187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:23.526012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 604/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:23.531998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:23.652858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4043ea-9398-4f16-8be4-ac41b9d03568 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:23.681037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:23.693222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:23.693843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:23.694121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:23.694404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:23.695108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:23.695408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:23.696824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:23.705413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 619/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:23.711508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:23.767427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:23.842249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36328e5-f347-4806-8fd5-4e1c7bc1b715 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:23.881160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:23.881968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:23.882254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:23.882534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:23.892624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:23.894078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:23.916799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:23.918066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:23.919475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:23.919475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:23.919475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:23.919475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:23.919475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 605/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:23.924571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:24.035532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb63219-1865-405d-addf-aff1978498e6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:24.069139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:24.069748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:24.070027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:24.070296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:24.070979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:24.071282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:24.074309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:24.086525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 620/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:24.092383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:24.191152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:24.198699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c62834-b8f8-463b-b11a-a251594d3df7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:24.241052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:24.241676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:24.241985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:24.242234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:24.250793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:24.252333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:03:24.263740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:24.264740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:24.266118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 606/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:24.271441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:24.321397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:24.374013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b97a6-45c1-4fc5-8aee-30d55e8b639d None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:24.409273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:24.409890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:24.410163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:24.410435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:24.411220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:24.411519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:24.412989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:24.430495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 621/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Fri Aug 5 05:03:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:24.436286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:24.541845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928aa3e-a892-4d1a-8340-2aa8ffbd5e2a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:24.574140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:24.574707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:24.574984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:24.575253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:24.586490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:24.587997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:24.601909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:24.603028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:24.604166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:24.604166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:24.604166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:24.604166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:24.604166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 607/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:24.608929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:24.710515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e014f-914c-4135-9df5-2ced9681bd08 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:24.712824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:24.747688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:24.748377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:24.748639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:24.748913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:24.749646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:24.749914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:24.751369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:24.763413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 622/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:24.769791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:24.780313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:24.873158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977005a-57c6-4fb9-8de6-069dbf682f02 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:24.910520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:24.911232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:24.911546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:24.911993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:24.919087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:24.920558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:03:24.932134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:24.933079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:24.933989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 608/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:03:24.939751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:25.058580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f153e-a458-4684-a1a7-79a65712a551 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:25.096342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:25.097041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:25.097397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:25.097668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:25.098527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:25.098835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:25.100530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699413b-0d6d-463d-9184-9060cb88098b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:25.114116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 623/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:03:25.120355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:25.189783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:25.228752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c2042-093b-48ab-be01-591db9bb1e53 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:25.263514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba437631-c8ae-4d79-860e-a1b6603375d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:25.264325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba437631-c8ae-4d79-860e-a1b6603375d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:25.264618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba437631-c8ae-4d79-860e-a1b6603375d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:25.264924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba437631-c8ae-4d79-860e-a1b6603375d7 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/7b3c6d4d4371427ca4ca74a962a47967/roles/6d6fa6b47348438cbe9d67a78855ef7e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:25.280433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba437631-c8ae-4d79-860e-a1b6603375d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=7b3c6d4d4371427ca4ca74a962a47967, role_id=6d6fa6b47348438cbe9d67a78855ef7e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:25.282100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba437631-c8ae-4d79-860e-a1b6603375d7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:25.299421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 609/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:03:25 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/7b3c6d4d4371427ca4ca74a962a47967/roles/6d6fa6b47348438cbe9d67a78855ef7e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:03:27.329256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff360-f938-4ba3-abb9-0fcd58869651 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:27.329773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff360-f938-4ba3-abb9-0fcd58869651 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:27.329773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff360-f938-4ba3-abb9-0fcd58869651 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:27.331133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 624/1233] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:27.337261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:27.337807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:27.337976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:27.358556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:27.359831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:27.429490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 610/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:03:27.453597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff360-f938-4ba3-abb9-0fcd58869651 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:27.481402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:27.567822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff360-f938-4ba3-abb9-0fcd58869651 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:27.609520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:27.610308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:27.610676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:27.611036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:27.622762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:03:27.628081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:27.629215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 05:03:27.630263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:27.634636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 625/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Fri Aug 5 05:03:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:27.636403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:27.686315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:27.743671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6fb7b-c7e0-4076-bfa2-c63437886218 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:27.791651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04895089-4247-49f8-8a07-582023752570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:27.792712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04895089-4247-49f8-8a07-582023752570 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:27.793239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04895089-4247-49f8-8a07-582023752570 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:27.793721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04895089-4247-49f8-8a07-582023752570 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:27.794715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04895089-4247-49f8-8a07-582023752570 None admin] RBAC: Adding query filter params (name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:27.795206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04895089-4247-49f8-8a07-582023752570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:27.797426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04895089-4247-49f8-8a07-582023752570 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:27.807812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 611/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Fri Aug 5 05:03:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:27.813754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:27.909037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:27.925721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addc498-7ae6-4e51-be57-3260829e71d6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:27.961686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:27.962339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:27.962621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:27.962893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:27.973641 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:27.975107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:27.993065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:27.994305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:27.995457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:27.995457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:27.995457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:27.995457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:27.996589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 626/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:28.002891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04895089-4247-49f8-8a07-582023752570 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:28.119797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04895089-4247-49f8-8a07-582023752570 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:28.129130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:28.155953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:28.156671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:28.156947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:28.157232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:28.157917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:28.158221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:28.159732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:28.171706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 612/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:28.177922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:28.282523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23943d8-e7be-4746-ae00-4797f53326a0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:28.318840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:28.319485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:28.319790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:28.320054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:28.322717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:28.328231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:28.329839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:03:28.341327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:28.343101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:28.344868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 627/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:28.348539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:28.462645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e19aac-936a-4b4b-ad42-affb9d993fb3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:28.498242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:28.498971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:28.499248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:28.499540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:28.500396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:28.500703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:28.502131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:28.520487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 613/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Fri Aug 5 05:03:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:28.526785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:28.639278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96412a9-e151-4eb5-a8b3-935e34a8678e None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:28.678706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:28.679329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:28.679611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:28.679923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:28.747840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:28.756092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:28.757642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:28.862774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 628/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:03:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:03:28.868937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:28.899019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:28.983214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545542b7-a7b0-4700-80fb-045c8fd010e4 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:29.019456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:29.020146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:29.020423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:29.020693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:29.033266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:29.034829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:29.052619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:29.053797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:29.054874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:29.054874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:29.054874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:29.054874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:29.054874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 614/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:29.060046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:29.164758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:29.172035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13665e-aae9-47b5-9eca-9bee67a158b6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:29.210884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:29.211510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:29.211809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:29.212159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:29.212848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:29.213159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:29.214683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:29.228818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 629/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:29.235646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:29.321829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:29.345359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd1260-17ad-4f83-9bc6-b6afcbf43ab7 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:29.379390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:29.380096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:29.380335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:29.380602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:29.391143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:29.392753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:03:29.405454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:29.406385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:29.408394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 615/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:29.412876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:29.528200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed145a2-fc3b-4e39-b669-1e2a01dce3f8 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:29.572586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:29.573178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:29.573492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:29.573871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:29.574874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:29.575210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:29.577043 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b647-d87a-40f7-a10b-4db8e184b75b None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:29.591066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 630/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:03:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:29.597738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:29.703491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105f83-2567-4428-9d62-e422db3b403f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:29.736728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:29.737389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:29.737670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:29.738002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:29.738808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] RBAC: Adding query filter params (role.id=6d6fa6b47348438cbe9d67a78855ef7e, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=7b3c6d4d4371427ca4ca74a962a47967) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:29.739121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:29.740699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea6ea8-83d9-400a-998d-4ad3797a61b6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:29.747352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 616/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:03:29 2022] GET /identity/v3/role_assignments?user.id=7b3c6d4d4371427ca4ca74a962a47967&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=6d6fa6b47348438cbe9d67a78855ef7e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:31.791675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f032-2b1f-45da-948b-57c60882a02f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:31.791675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f032-2b1f-45da-948b-57c60882a02f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:31.792767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f032-2b1f-45da-948b-57c60882a02f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:31.794545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 631/1247] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:31.800679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:31.801243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:31.801243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:31.807905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:31.820918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:31.822113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:31.888249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 617/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 05:03:31.911488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832f032-2b1f-45da-948b-57c60882a02f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:31.929950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:32.020239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832f032-2b1f-45da-948b-57c60882a02f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:32.059794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:32.060378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:32.060660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:32.060936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:32.072083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:03:32.077185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:32.078483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:32.079313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 632/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:03:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:32.085177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:32.194945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c79b5-75e9-4a46-b876-eac7d6217a11 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:32.226688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:32.232847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:32.233479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:32.233756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:32.234025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:32.234830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:32.235137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:32.236570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:32.245393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 618/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:32.251657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:32.335199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:32.359101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625eeec4-dea0-4b8e-ba89-c4540cee4a9a None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:32.393159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:32.393797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:32.394075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:32.394350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:32.405012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:32.406615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:32.423956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:32.425517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:32.426635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:32.426635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:32.426635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:32.426635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:32.431790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 633/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:32.431940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:32.536981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b1bca-4420-453a-bfef-757048d4fff2 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:32.573133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:32.573743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:32.574019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:32.574343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:32.574969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:32.575274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:32.576683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:32.588542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 619/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:32.595080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:32.702606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3fe08-49cb-4b83-8e09-9d985872f261 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:32.737088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:32.746675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:32.748031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:32.748626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:32.748867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:32.758684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:32.760526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:03:32.774788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:32.775946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:32.776918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 634/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 05:03:32.783052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:32.848612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:32.899433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2457-cfa9-41a3-9155-cc8a0d984b0f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:32.939069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:32.939710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:32.940011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:32.940285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:32.941084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:32.941388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:32.942817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:32.961878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 620/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Fri Aug 5 05:03:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:32.968340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:33.085403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c9738-eec6-4ecd-9ba7-1b7af119ec4f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:33.125562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:33.126195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:33.126494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:33.126813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:33.195320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:33.208492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:33.210170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:33.316573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 635/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:03:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:03:33.322682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:33.438201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a78bb-fbac-436b-89f1-3eed1a68e2ef None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:33.453470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:33.479266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:33.480020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:33.480314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:33.480610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:33.492149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:33.493620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:33.511210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:33.512390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:33.513466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:33.513466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:33.513466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:33.513466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:33.513466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 621/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:33.519379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:33.621670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:33.635098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45811a-703f-4fcd-938b-b5f156cfffd0 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:33.671473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:33.672165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:33.672443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:33.672713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:33.673401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:33.673753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:33.675601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:33.687814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 636/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:33.693769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:33.812147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c49b06-6e96-4d19-9800-ba847e9ba87f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:33.851581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:33.852342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:33.852616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:33.852846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:33.860416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:33.861961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:03:33.874879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:33.876917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:33.877846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 622/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:33.883871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:33.995050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef017b2-66d9-46c1-9eca-db2bbe1f4f81 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:34.037083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:34.037743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:34.038022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:34.038291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:34.039033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:34.039335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:34.040886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62736fc4-9442-4d5b-8c0b-be87fefa11ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:34.056619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 637/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:34.063054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:34.089708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:34.168145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6aad8-2af5-43f0-a394-be3426472c57 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:34.206932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:34.207595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:34.207953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:34.208260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:34.209074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=7b3c6d4d4371427ca4ca74a962a47967) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:34.209387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:34.211014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdac89-fcee-4b03-b4cc-841a65b629e6 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:34.217055 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 623/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:03:34 2022] GET /identity/v3/role_assignments?user.id=7b3c6d4d4371427ca4ca74a962a47967&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=47aca6aac2934a5cb816872abfc94994 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:36.213632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b9b3-d5c1-4957-87b9-b42f2c788c60 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:36.214199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b9b3-d5c1-4957-87b9-b42f2c788c60 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:36.214199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b9b3-d5c1-4957-87b9-b42f2c788c60 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:36.215512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 638/1261] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:36.221220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:36.221674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:36.221763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:36.240879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:36.242111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:36.311070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 624/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:36.333329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0b9b3-d5c1-4957-87b9-b42f2c788c60 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:36.344944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:36.442904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0b9b3-d5c1-4957-87b9-b42f2c788c60 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:36.477800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:36.478432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:36.478709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:36.478980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:36.489263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:03:36.494133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:36.495238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:36.496476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 639/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:36.503099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:36.532032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:36.616298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca33189-ed6e-48c9-a6e9-8974de77cda3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:36.651194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:36.651812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:36.652110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:36.652380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:36.653092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:36.653394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:36.654753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:36.662572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 625/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:36.669010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:36.749785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:36.789889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d601-6ea1-4b46-91b3-658ae9165a62 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:36.825271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:36.825911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:36.826189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:36.826458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:36.836365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:36.837817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:36.852060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:36.853032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:36.853950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:36.853950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:36.853950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:36.853950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:36.853950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 640/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:36.859435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:36.944169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:36.974410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c5541-0aa3-40c2-8db6-02944dd8998e None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:37.013377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:37.014012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:37.014293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:37.014558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:37.015248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:37.015629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:37.017051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:37.029183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 626/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:37.034966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:37.150902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f1871-2d8c-417d-8542-2b45588503ab None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:37.173475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:37.189847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:37.190491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:37.190772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:37.191040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:37.198894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:37.200334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:03:37.211705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:37.212824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:37.218784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 641/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:37.220554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:37.321898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca000b-ff42-4033-86cc-fe51b2bf6508 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:37.361061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:37.361665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:37.361939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:37.362209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:37.362993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:37.363299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:37.364737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:37.374343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:37.383505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 627/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Fri Aug 5 05:03:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:37.389915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:37.499900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a774c-bef9-410e-ae0c-b5bbad031e16 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:37.534427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:37.535051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:37.535328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:37.535598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:37.546427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:37.547962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:37.562659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:37.563648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:37.564786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:37.564786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:37.564786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:37.564786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:37.564786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 642/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:37.572175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:37.677187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317769e7-9370-4a48-80e9-1601efbf69cd None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:37.712734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:37.713358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:37.713667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:37.713936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:37.714628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:37.714935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:37.716370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:37.729042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 628/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:37.737243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:37.777113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:37.850297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8bca4-efcd-4509-878e-5dc0c62cb0ce None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:37.885981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:37.886612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:37.886853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:37.887154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:37.894106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:37.895546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:03:37.907722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:37.908748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:37.910013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 643/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:37.918266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:37.982475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:38.026504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6c96-f75b-437d-bbe8-b7ca75bc068c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:38.063611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:38.064187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:38.064464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:38.064733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:38.065474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:38.065785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:38.067207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2d70e-91e9-434b-9563-a73161e9a22a None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:38.079391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 629/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:38.086172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:38.189555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:38.200652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f34d1a-01eb-4336-8731-e8c5788e50f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:38.235331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68ce56-576c-4e78-9c0d-148651a906e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:38.235998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68ce56-576c-4e78-9c0d-148651a906e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:38.236292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68ce56-576c-4e78-9c0d-148651a906e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:38.236562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68ce56-576c-4e78-9c0d-148651a906e4 None admin] PATH_INFO: `/v3/projects/434c02e53e1341a49390f0aec50efc19/users/7b3c6d4d4371427ca4ca74a962a47967/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:38.238524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68ce56-576c-4e78-9c0d-148651a906e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=434c02e53e1341a49390f0aec50efc19, user_id=7b3c6d4d4371427ca4ca74a962a47967, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:38.239988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68ce56-576c-4e78-9c0d-148651a906e4 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:38.258759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 644/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Fri Aug 5 05:03:38 2022] PUT /identity/v3/projects/434c02e53e1341a49390f0aec50efc19/users/7b3c6d4d4371427ca4ca74a962a47967/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:03:40.206490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600150c9-be93-4211-a2c1-73c0b42f2b4a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:40.207063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600150c9-be93-4211-a2c1-73c0b42f2b4a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:40.207063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600150c9-be93-4211-a2c1-73c0b42f2b4a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:40.209303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 630/1274] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:40.217055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:40.217510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:40.217597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:40.236269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:40.237453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:40.306860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 645/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:40.334476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600150c9-be93-4211-a2c1-73c0b42f2b4a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:40.441468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600150c9-be93-4211-a2c1-73c0b42f2b4a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:40.458655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:40.476431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:40.477056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:40.477330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:40.477597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:40.487500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 05:03:40.492303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:40.493371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:40.494226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 631/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Fri Aug 5 05:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:40.499486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:40.585380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:40.605430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f31c7-b962-4e05-9fe8-035284655028 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:40.644833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:40.645449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:40.645733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:40.646009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:40.646728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:40.647038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:40.648442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:40.656551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 646/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Fri Aug 5 05:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:40.662565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:40.772155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dad9d-e76f-49ac-874d-2f4a473d7a3a None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:40.806572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:40.807282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:40.807683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:40.808119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:40.821584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:40.823127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:40.837769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:40.838761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:40.839898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:40.839898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:40.839898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:40.839898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:40.839898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 632/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:40.845019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:40.958595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02352c7-bf92-4442-a9ff-1cea8513d424 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:40.993374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:40.993988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:40.994270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:40.994542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:40.995234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:40.995538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:40.996951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:41.009180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 647/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:41.015161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:41.047872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:41.120488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd30bd0-7c75-4b88-83f7-91bfc566db19 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:41.155893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:41.156576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:41.156854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:41.157122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:41.164741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:41.166179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 05:03:41.177833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:41.178843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:41.180022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 633/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:41.184932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:41.189117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:41.290678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac45a57-c8be-4455-ab10-6d665effc102 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:41.326237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:41.326848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:41.327166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:41.327443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:41.328476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:41.328783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:41.330227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:41.347197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 648/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Fri Aug 5 05:03:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:41.356238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:41.451322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:41.462694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b55b0-c07c-4405-9e35-46bc77def68c None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:41.498074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:41.498681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:41.498966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:41.499237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:41.568811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:41.570251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:41.671363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:41.682594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 634/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Fri Aug 5 05:03:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 05:03:41.689050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:41.756250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:41.796531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a921-d04d-4003-8d17-4780724a259f None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:41.831688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:41.832355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:41.832630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:41.832907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:41.843182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:41.844761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 05:03:41.859413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:41.860493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:41.861413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 05:03:41.861413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 05:03:41.861413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 05:03:41.861413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:41.861413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 649/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:41.866407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:41.973203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f12a2-8667-4d9a-aa14-af23627869ac None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:42.010374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:42.010954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:42.011235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:42.011506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:42.012225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:42.012529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:42.013884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:42.025724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 635/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Fri Aug 5 05:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:03:42.031582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:42.135225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4199e5-240b-4129-96e8-9fb544871d1c None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:42.149137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:42.170822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:42.171453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:42.171734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:42.172038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:42.178984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:42.180470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 05:03:42.191560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:42.192622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:03:42.193653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 650/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Fri Aug 5 05:03:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:42.199449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:42.268374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:42.325107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db299aa-3245-4d8b-8a1c-9dcc7a0d7bf3 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:42.359251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:42.359906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:42.360174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:42.360473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:42.361242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:42.361548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:42.362974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967da4f-0359-45d7-bd7f-7e357e8cab69 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:42.374961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 636/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Fri Aug 5 05:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:42.381993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:42.487347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63939a67-968b-409f-983d-05cbeed0c8f6 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:42.522640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:42.523180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:42.523460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:42.523733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:42.524556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] RBAC: Adding query filter params (role.id=47aca6aac2934a5cb816872abfc94994, scope.project.id=434c02e53e1341a49390f0aec50efc19, user.id=7b3c6d4d4371427ca4ca74a962a47967) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:42.524865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:42.526347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c25406-dedd-4dd2-b31f-75e3a38c6ee9 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:42.532402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 651/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Fri Aug 5 05:03:42 2022] GET /identity/v3/role_assignments?user.id=7b3c6d4d4371427ca4ca74a962a47967&scope.project.id=434c02e53e1341a49390f0aec50efc19&role.id=47aca6aac2934a5cb816872abfc94994 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:44.535179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47664271-3340-43b9-9752-3884061d33d1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:44.535800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47664271-3340-43b9-9752-3884061d33d1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:44.535800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47664271-3340-43b9-9752-3884061d33d1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:44.537128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 637/1288] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:44.543379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:44.544088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:44.544357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:44.570910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:44.572107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:44.602163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:44.649683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 652/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:44.671989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47664271-3340-43b9-9752-3884061d33d1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:44.784923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47664271-3340-43b9-9752-3884061d33d1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:44.797196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:44.823003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:44.823650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:44.823963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:44.824242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:44.824875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:44.826280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:44.835087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:44.836253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:44.837510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 638/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:03:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:44.844813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:44.944790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f151c-6045-45eb-b8eb-742c808c99d3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:44.981158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:44.981762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:44.982044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:44.982314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:44.982992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:44.983291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:44.984727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e0adf-da20-40eb-9e1b-18fd6a4abf22 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:44.989692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 653/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Fri Aug 5 05:03:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:44.995473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:45.109290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f161f-b3db-4a78-895f-1ef63871598f None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:45.148679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:45.149288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:45.149559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:45.149828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] PATH_INFO: `/v3/services` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:45.150496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:45.150802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:45.152192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e64f39-51d2-4874-9c57-35c5a951674b None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:45.157249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 639/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Fri Aug 5 05:03:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:47.088226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bfc5b-0f53-463f-9c06-f9317a0f579e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:47.089045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bfc5b-0f53-463f-9c06-f9317a0f579e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:47.089045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bfc5b-0f53-463f-9c06-f9317a0f579e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:47.091279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 654/1293] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:47.097474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3cc9-ac67-4ada-bfe3-a27b5d7b356d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:47.098166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3cc9-ac67-4ada-bfe3-a27b5d7b356d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:47.098283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3cc9-ac67-4ada-bfe3-a27b5d7b356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:47.128164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-51bb3cc9-ac67-4ada-bfe3-a27b5d7b356d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:47.129347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-51bb3cc9-ac67-4ada-bfe3-a27b5d7b356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:47.161563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:47.206692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 640/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:47.241283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bfc5b-0f53-463f-9c06-f9317a0f579e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:47.286364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:47.376916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bfc5b-0f53-463f-9c06-f9317a0f579e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:47.411827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f066a-c31f-4c31-99af-dc56cecf327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:47.412468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f066a-c31f-4c31-99af-dc56cecf327c None admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:47.412754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f066a-c31f-4c31-99af-dc56cecf327c None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:47.413029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f066a-c31f-4c31-99af-dc56cecf327c None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:47.413725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f066a-c31f-4c31-99af-dc56cecf327c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:47.415103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f066a-c31f-4c31-99af-dc56cecf327c None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:47.432359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 655/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Fri Aug 5 05:03:47 2022] POST /identity/v3/services => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:03:49.382983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a07c7-289f-4559-8a2b-d234d7ae2ffe None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:49.383568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a07c7-289f-4559-8a2b-d234d7ae2ffe None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:49.383568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a07c7-289f-4559-8a2b-d234d7ae2ffe None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:49.384852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 641/1296] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:49.391232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:49.391707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:49.391841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:49.416728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:49.417925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:49.494151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 656/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:49.517363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a07c7-289f-4559-8a2b-d234d7ae2ffe None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:49.628969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a07c7-289f-4559-8a2b-d234d7ae2ffe None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:49.667656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:49.668316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:49.668603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:49.668875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] PATH_INFO: `/v3/services/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:49.669498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:49.670870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:49.680361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:49.681632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:49.682550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 642/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:03:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:49.689541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:49.714135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:49.804965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d911dd0-82f3-4cce-91b1-127af60ddf0a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:49.845582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:49.846214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:49.846501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:49.846773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:49.847455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:49.847806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:49.849203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:49.854736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 657/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Fri Aug 5 05:03:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:49.861073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:49.893737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:49.968693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadf3fd-7f1a-406d-ad7e-9ac615fc56ed None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:50.007242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:50.007897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:50.008177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:50.008449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:50.009205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] RBAC: Adding query filter params (interface=public, service_id=b537a62e9ab5444dad4cd984275aa062, region_id=RegionOne) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:50.009512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:50.010929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a09f-d4ee-46c5-9ba1-56d719b7a870 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:50.016482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 643/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Fri Aug 5 05:03:49 2022] GET /identity/v3/endpoints?service_id=b537a62e9ab5444dad4cd984275aa062&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:50.022327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:50.108076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:50.128065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73251cea-c5e4-4d65-9a67-93ba60d44596 None admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:50.164041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e4c5-8a16-4357-8031-4c2b62e32725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:50.164656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e4c5-8a16-4357-8031-4c2b62e32725 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:50.164934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e4c5-8a16-4357-8031-4c2b62e32725 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:50.165203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e4c5-8a16-4357-8031-4c2b62e32725 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:50.165846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e4c5-8a16-4357-8031-4c2b62e32725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:50.167211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e4c5-8a16-4357-8031-4c2b62e32725 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:50.174283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 658/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Fri Aug 5 05:03:50 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:03:52.118838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caeb69-9a10-4272-b5d9-6d67c1f7944e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:52.119418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caeb69-9a10-4272-b5d9-6d67c1f7944e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:52.119418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66caeb69-9a10-4272-b5d9-6d67c1f7944e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:52.120777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 644/1302] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:03:52.127191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:52.127954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:52.128103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:52.157785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:03:52.158987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:03:52.228062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 659/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:03:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 05:03:52.259650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66caeb69-9a10-4272-b5d9-6d67c1f7944e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:52.357047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:52.365962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66caeb69-9a10-4272-b5d9-6d67c1f7944e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:52.400442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:52.401068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:52.401341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:52.401615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:52.402230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:52.403596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:03:52.412662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:03:52.413781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application with Lock( Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: ERROR keystone.server.flask.application Aug 05 05:03:52.414961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 645/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Fri Aug 5 05:03:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 05:03:52.420504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:52.500902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:03:52.526433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aeb55-6282-4d17-9e55-5d5a78b1c39f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:52.562112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:52.562722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:52.563004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:52.563304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] PATH_INFO: `/v3/services` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:52.564014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:03:52.564313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:52.565694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029d693-65ee-419e-803a-cc8beb3ba431 None admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:52.571035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 660/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Fri Aug 5 05:03:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:03:52.579945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:03:52.683147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af4ed3-4eca-4b48-9a7b-d200b4db6066 None admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:03:52.718817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218a57-1af7-4a1e-ac06-3245846e8a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:03:52.719500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218a57-1af7-4a1e-ac06-3245846e8a26 None admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:03:52.719810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218a57-1af7-4a1e-ac06-3245846e8a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:03:52.720077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218a57-1af7-4a1e-ac06-3245846e8a26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:03:52.720773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36218a57-1af7-4a1e-ac06-3245846e8a26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:03:52.722130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36218a57-1af7-4a1e-ac06-3245846e8a26 None admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:03:52.743341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 646/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:03:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:04:02.567875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=30 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:04:02.738841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:04:55.039053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99ba70-f59b-4c2a-bd21-e2bf3973c075 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:55.039604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99ba70-f59b-4c2a-bd21-e2bf3973c075 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:55.039604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99ba70-f59b-4c2a-bd21-e2bf3973c075 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:55.044285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 661/1307] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:04:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:04:55.046118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd583d27-8e43-4ba7-bd55-90601142a395 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:55.046580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd583d27-8e43-4ba7-bd55-90601142a395 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:55.046684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd583d27-8e43-4ba7-bd55-90601142a395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:55.120753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-bd583d27-8e43-4ba7-bd55-90601142a395 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:04:55.122434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bd583d27-8e43-4ba7-bd55-90601142a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:55.263805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 647/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:04:55.302290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0f7b-1d7c-482a-a3ac-245a9b81b43c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:55.302604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0f7b-1d7c-482a-a3ac-245a9b81b43c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:55.302808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0f7b-1d7c-482a-a3ac-245a9b81b43c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:55.304217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 662/1309] 10.222.0.30 () {54 vars in 965 bytes} [Fri Aug 5 05:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:04:55.309983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df378-fa5b-4af4-a3b0-14b0057471fa None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:55.310276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df378-fa5b-4af4-a3b0-14b0057471fa None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:55.310557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df378-fa5b-4af4-a3b0-14b0057471fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:55.431321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6c3df378-fa5b-4af4-a3b0-14b0057471fa None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:04:55.433822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6c3df378-fa5b-4af4-a3b0-14b0057471fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:55.479688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:04:55.543283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 648/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Fri Aug 5 05:04:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:04:55.552458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0f7b-1d7c-482a-a3ac-245a9b81b43c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:04:55.554286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c60c0f7b-1d7c-482a-a3ac-245a9b81b43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:55.672710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0f7b-1d7c-482a-a3ac-245a9b81b43c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:04:55.756710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:04:55.759927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:55.760276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:55.760622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:55.762328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:55.819172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:04:55.877786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:04:55.879331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418618e-5784-4f7c-bdd3-09f3cd700a31 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:04:55.899644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 663/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:04:58.176553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254a21a-ccd0-460d-b9f5-1e2a9351f368 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:58.177097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254a21a-ccd0-460d-b9f5-1e2a9351f368 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:58.177097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254a21a-ccd0-460d-b9f5-1e2a9351f368 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:58.178652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 649/1312] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:04:58.183748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb432e1-2014-4648-b9b7-00b040277f07 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:58.184213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb432e1-2014-4648-b9b7-00b040277f07 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:58.184428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb432e1-2014-4648-b9b7-00b040277f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:58.267441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bbb432e1-2014-4648-b9b7-00b040277f07 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:04:58.268585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bbb432e1-2014-4648-b9b7-00b040277f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:58.371814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 664/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:04:58.408093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254a21a-ccd0-460d-b9f5-1e2a9351f368 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:04:58.409867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2254a21a-ccd0-460d-b9f5-1e2a9351f368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:58.533721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254a21a-ccd0-460d-b9f5-1e2a9351f368 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:04:58.613808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:04:58.616595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:04:58.617259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:04:58.617743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:04:58.618402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:04:58.620688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:04:58.736736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:04:58.738228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2463971-3da2-44ea-80ec-bd3564e846b5 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:04:58.759078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 650/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:01.061897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e5faa-59d7-4e84-b4ba-5028aca9bc58 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:01.062424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e5faa-59d7-4e84-b4ba-5028aca9bc58 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:01.062424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e5faa-59d7-4e84-b4ba-5028aca9bc58 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:01.063818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 665/1315] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:01.070271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2458-a431-4e17-bc64-667709f0737a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:01.070738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2458-a431-4e17-bc64-667709f0737a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:01.070833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2458-a431-4e17-bc64-667709f0737a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:01.146556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a25e2458-a431-4e17-bc64-667709f0737a None None] MFA Rules not processed for user `8d8d9ef9df2d4b99998380797a17d038`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:01.147674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a25e2458-a431-4e17-bc64-667709f0737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:01.205549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:01.250514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 651/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:01.317337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043270e-e248-45d1-b107-e6e6d43d3f2c None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:01.317733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043270e-e248-45d1-b107-e6e6d43d3f2c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:01.318137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043270e-e248-45d1-b107-e6e6d43d3f2c None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:01.320402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 666/1317] 10.222.0.30 () {54 vars in 965 bytes} [Fri Aug 5 05:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:01.327058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a6f98-7d7e-40c0-af25-4730e0555447 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:01.327471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a6f98-7d7e-40c0-af25-4730e0555447 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:01.327903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a6f98-7d7e-40c0-af25-4730e0555447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:01.416449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c86a6f98-7d7e-40c0-af25-4730e0555447 None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:01.417695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c86a6f98-7d7e-40c0-af25-4730e0555447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:01.505486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 652/1318] 10.222.0.30 () {58 vars in 1062 bytes} [Fri Aug 5 05:05:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:01.519510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043270e-e248-45d1-b107-e6e6d43d3f2c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:01.521102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c043270e-e248-45d1-b107-e6e6d43d3f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:01.531993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:01.632551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043270e-e248-45d1-b107-e6e6d43d3f2c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:01.710007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:01.710556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:01.710846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:01.711114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:01.712776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:01.777937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:01.829962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:01.831418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85d8dd-67f0-428c-9549-7f80f45560ea service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:01.851627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 667/1319] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:04.774602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686951-9c79-445d-8559-ef2cba3e429f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:04.775462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686951-9c79-445d-8559-ef2cba3e429f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:04.775462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686951-9c79-445d-8559-ef2cba3e429f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:04.777836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 653/1320] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:04.783904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da8459-2442-441f-8a29-971fcb363bec None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:04.784341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da8459-2442-441f-8a29-971fcb363bec None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:04.784469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da8459-2442-441f-8a29-971fcb363bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:04.863361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-a3da8459-2442-441f-8a29-971fcb363bec None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:04.865264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a3da8459-2442-441f-8a29-971fcb363bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:04.976931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 668/1321] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:05.016086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686951-9c79-445d-8559-ef2cba3e429f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:05.017688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-63686951-9c79-445d-8559-ef2cba3e429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:05.071256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:05.126394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686951-9c79-445d-8559-ef2cba3e429f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:05.201604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:05.202233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:05.202515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:05.202784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:05.204462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:05.305246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:05.326952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:05.329052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b5075-6ff1-4cab-9d8c-2d5257835548 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:05.348129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 654/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:08.032312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6a3e4-cc43-4654-b8c3-5900ce3ae120 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:08.032913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6a3e4-cc43-4654-b8c3-5900ce3ae120 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:08.033018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6a3e4-cc43-4654-b8c3-5900ce3ae120 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:08.034328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 669/1323] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:08.039630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40a428-671c-43b5-8a8c-27f8b601546c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:08.040080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40a428-671c-43b5-8a8c-27f8b601546c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:08.040263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40a428-671c-43b5-8a8c-27f8b601546c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:08.105048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ef40a428-671c-43b5-8a8c-27f8b601546c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:08.106265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ef40a428-671c-43b5-8a8c-27f8b601546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:08.212987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 655/1324] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:08.252084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6a3e4-cc43-4654-b8c3-5900ce3ae120 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:08.254190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6cd6a3e4-cc43-4654-b8c3-5900ce3ae120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:08.281077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:08.371162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6a3e4-cc43-4654-b8c3-5900ce3ae120 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:08.448505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:08.449133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:08.449406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:08.449647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:08.451295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:08.523501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:08.562732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:08.564317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66684c-70b2-47a7-9269-40284dcd8556 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:08.582281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 670/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:10.619517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062abd2e-5e1a-4b93-95bb-5fbc29191d03 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:10.620079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062abd2e-5e1a-4b93-95bb-5fbc29191d03 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:10.620079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062abd2e-5e1a-4b93-95bb-5fbc29191d03 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:10.621363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 656/1326] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:10.626775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa572a-0156-4d9a-b861-e7369347c9aa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:10.627230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa572a-0156-4d9a-b861-e7369347c9aa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:10.627326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa572a-0156-4d9a-b861-e7369347c9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:10.696331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2caa572a-0156-4d9a-b861-e7369347c9aa None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:10.697530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2caa572a-0156-4d9a-b861-e7369347c9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:10.796262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 671/1327] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:10.832141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062abd2e-5e1a-4b93-95bb-5fbc29191d03 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:10.833707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-062abd2e-5e1a-4b93-95bb-5fbc29191d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:10.898236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:10.945433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062abd2e-5e1a-4b93-95bb-5fbc29191d03 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:11.041337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:11.041965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:11.042272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:11.042537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:11.044184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:11.156153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:11.158700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:11.160355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8689f60-e3a9-4295-9aef-8d60e52a001f service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:11.178871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 657/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:13.229757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e683-67fa-4d6d-9564-313079231cd7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:13.230355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e683-67fa-4d6d-9564-313079231cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:13.230355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e683-67fa-4d6d-9564-313079231cd7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:13.231673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 672/1329] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:13.237072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f8e4-f2bf-4d28-b055-7877f0a635d5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:13.237541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f8e4-f2bf-4d28-b055-7877f0a635d5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:13.237650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f8e4-f2bf-4d28-b055-7877f0a635d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:13.314440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7191f8e4-f2bf-4d28-b055-7877f0a635d5 None None] MFA Rules not processed for user `8d8d9ef9df2d4b99998380797a17d038`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:13.315664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7191f8e4-f2bf-4d28-b055-7877f0a635d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:13.419662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 658/1330] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:13.457501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308e683-67fa-4d6d-9564-313079231cd7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:13.459094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d308e683-67fa-4d6d-9564-313079231cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:13.490680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:13.566385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308e683-67fa-4d6d-9564-313079231cd7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:13.644958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:13.645611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:13.645888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:13.646157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:13.647796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:13.725778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:13.779284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:13.782078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3977a0-dc6a-44fa-bf07-9fcdc9b3b7ee service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:13.803045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 673/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:16.062595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1a38c-c7d2-4855-90d3-e408f55f0725 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:16.063135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1a38c-c7d2-4855-90d3-e408f55f0725 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:16.063135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1a38c-c7d2-4855-90d3-e408f55f0725 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:16.064624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 659/1332] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:16.073347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a7f2-5598-498f-8ee3-0f4ede37f764 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:16.074108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a7f2-5598-498f-8ee3-0f4ede37f764 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:16.074213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a7f2-5598-498f-8ee3-0f4ede37f764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:16.149569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e925a7f2-5598-498f-8ee3-0f4ede37f764 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:16.150789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e925a7f2-5598-498f-8ee3-0f4ede37f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:16.254334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 674/1333] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:16.291367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1a38c-c7d2-4855-90d3-e408f55f0725 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:16.293833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-85b1a38c-c7d2-4855-90d3-e408f55f0725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:16.384919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:16.421125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1a38c-c7d2-4855-90d3-e408f55f0725 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:16.501168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:16.501816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:16.502098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:16.502387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:16.504042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:16.621223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:16.622580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ae231-968b-4839-98a0-add2d392e6a3 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:16.633498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:16.643537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 660/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:18.685219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a4e98-feb0-42d6-a536-a446d9faf02a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:18.686118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a4e98-feb0-42d6-a536-a446d9faf02a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:18.686118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a4e98-feb0-42d6-a536-a446d9faf02a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:18.688305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 675/1335] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:18.694423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dcfb8-d4f0-4d42-a9a5-386440542481 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:18.694853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dcfb8-d4f0-4d42-a9a5-386440542481 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:18.694992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dcfb8-d4f0-4d42-a9a5-386440542481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:18.769649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9f1dcfb8-d4f0-4d42-a9a5-386440542481 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:18.770863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9f1dcfb8-d4f0-4d42-a9a5-386440542481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:18.875111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 661/1336] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:18.921783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a4e98-feb0-42d6-a536-a446d9faf02a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:18.923438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-422a4e98-feb0-42d6-a536-a446d9faf02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:18.979371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:19.047176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a4e98-feb0-42d6-a536-a446d9faf02a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:19.123080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:19.123803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:19.124085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:19.124372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:19.125991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:19.216799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:19.244812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:19.246164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d872d-20a3-4a57-bf6e-8c3d1bd933f8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:19.264281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 676/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:21.623036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068614a-3590-4bd1-a425-dfba21351fb0 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:21.623672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068614a-3590-4bd1-a425-dfba21351fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:21.623672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068614a-3590-4bd1-a425-dfba21351fb0 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:21.625019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 662/1338] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:21.631406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ce06-d5bd-43bc-bf0a-475dc5931b94 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:21.632215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ce06-d5bd-43bc-bf0a-475dc5931b94 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:21.632384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ce06-d5bd-43bc-bf0a-475dc5931b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:21.703926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-82d7ce06-d5bd-43bc-bf0a-475dc5931b94 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:21.705216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-82d7ce06-d5bd-43bc-bf0a-475dc5931b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:21.812245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 677/1339] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:21.854500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068614a-3590-4bd1-a425-dfba21351fb0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:21.856081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7068614a-3590-4bd1-a425-dfba21351fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:21.935349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:21.971470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068614a-3590-4bd1-a425-dfba21351fb0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:22.049223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:22.049855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:22.050123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:22.050394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:22.052066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:22.178591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:22.180062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db3452-1b7d-4273-86a4-1ce07ee9012f service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:22.198465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 663/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:26.167684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f7e43-33e7-4af0-b12e-c49a0749320f None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:26.168300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f7e43-33e7-4af0-b12e-c49a0749320f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:26.168300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f7e43-33e7-4af0-b12e-c49a0749320f None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:26.169621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 678/1341] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:26.175388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab802c21-3a11-4799-82db-aa008cbc72ec None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:26.175859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab802c21-3a11-4799-82db-aa008cbc72ec None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:26.175962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab802c21-3a11-4799-82db-aa008cbc72ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:26.181672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:26.244557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ab802c21-3a11-4799-82db-aa008cbc72ec None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:26.245773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ab802c21-3a11-4799-82db-aa008cbc72ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:26.345924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 664/1342] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:26.384987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f7e43-33e7-4af0-b12e-c49a0749320f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:26.386597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-724f7e43-33e7-4af0-b12e-c49a0749320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:26.436623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:26.495969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f7e43-33e7-4af0-b12e-c49a0749320f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:26.572821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:26.573479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:26.573763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:26.574038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:26.575750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:26.669465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:26.687239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:26.688699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8043d09-afa4-42a4-9bb9-4c02b2856f35 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:26.707199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 679/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:29.484688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2af8d-9825-4228-9e74-4fded210df5c None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:29.485297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2af8d-9825-4228-9e74-4fded210df5c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:29.485297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2af8d-9825-4228-9e74-4fded210df5c None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:29.486682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 665/1344] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:29.492037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a517a-5aa8-4354-bdb3-72f23fc9891e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:29.492511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a517a-5aa8-4354-bdb3-72f23fc9891e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:29.492719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a517a-5aa8-4354-bdb3-72f23fc9891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:29.561986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-205a517a-5aa8-4354-bdb3-72f23fc9891e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:29.563216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-205a517a-5aa8-4354-bdb3-72f23fc9891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:29.664372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 680/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:29.702417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2af8d-9825-4228-9e74-4fded210df5c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:29.704089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40f2af8d-9825-4228-9e74-4fded210df5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:29.796387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:29.814882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2af8d-9825-4228-9e74-4fded210df5c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:29.890786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:29.891424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:29.891705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:29.892183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:29.893835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:30.016699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:30.018279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19960c7c-6576-4cda-b89d-388543c0db12 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:30.037520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 666/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:34.068360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ae6d6-17b1-4557-bcef-3fd209d146f6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:34.068850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ae6d6-17b1-4557-bcef-3fd209d146f6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:34.068944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ae6d6-17b1-4557-bcef-3fd209d146f6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:34.070317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 681/1347] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:34.076042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978bdb0-4275-4f86-9262-813c8230e62d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:34.076546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978bdb0-4275-4f86-9262-813c8230e62d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:34.076546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978bdb0-4275-4f86-9262-813c8230e62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:34.092834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:34.145691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0978bdb0-4275-4f86-9262-813c8230e62d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:34.146913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0978bdb0-4275-4f86-9262-813c8230e62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:34.249079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 667/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:34.287277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ae6d6-17b1-4557-bcef-3fd209d146f6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:34.289045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9c0ae6d6-17b1-4557-bcef-3fd209d146f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:34.351942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:34.400041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ae6d6-17b1-4557-bcef-3fd209d146f6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:34.476905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:34.477576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:34.477857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:34.478128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:34.479840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:34.593450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:34.597777 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:34.599248 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac856e40-bff4-424b-82c8-c6a766e661a7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:34.617555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 682/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:39.509829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dd825-7d2f-48f0-8f17-9a9a62de1531 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:39.509829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dd825-7d2f-48f0-8f17-9a9a62de1531 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:39.510426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dd825-7d2f-48f0-8f17-9a9a62de1531 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:39.512367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 668/1350] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:05:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:39.516834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6614a-5ba4-4e93-984e-e166a8294f71 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:39.517217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6614a-5ba4-4e93-984e-e166a8294f71 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:39.517401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6614a-5ba4-4e93-984e-e166a8294f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:39.653543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b2f6614a-5ba4-4e93-984e-e166a8294f71 None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:39.654696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b2f6614a-5ba4-4e93-984e-e166a8294f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:39.771354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 683/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:05:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:43.751992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29739b-34cc-46c9-afd4-b4a2803ad3e1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:43.752938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29739b-34cc-46c9-afd4-b4a2803ad3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:43.753498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29739b-34cc-46c9-afd4-b4a2803ad3e1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:43.755522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 669/1352] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:43.761594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400daa-a8ed-45b4-b652-76255bc33281 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:43.762038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400daa-a8ed-45b4-b652-76255bc33281 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:43.762214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400daa-a8ed-45b4-b652-76255bc33281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:43.821470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:43.846154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2e400daa-a8ed-45b4-b652-76255bc33281 None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:43.847579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2e400daa-a8ed-45b4-b652-76255bc33281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:43.960598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 684/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:44.247284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=59 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:45.351968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf48b0-88a8-41fc-954c-6388725b93d2 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:45.352619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf48b0-88a8-41fc-954c-6388725b93d2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:45.352619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf48b0-88a8-41fc-954c-6388725b93d2 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:45.353911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 670/1354] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:45.359165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dba238-d0c3-4f64-ab4a-d2532ce96b7e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:45.359584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dba238-d0c3-4f64-ab4a-d2532ce96b7e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:45.359991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dba238-d0c3-4f64-ab4a-d2532ce96b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:45.440314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f5dba238-d0c3-4f64-ab4a-d2532ce96b7e None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:45.441525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5dba238-d0c3-4f64-ab4a-d2532ce96b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:45.486474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:45.553973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 685/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:05:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:05:48.932041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924068d-6cda-45bf-9a42-62638fbf413c None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:48.933016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924068d-6cda-45bf-9a42-62638fbf413c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:48.933499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924068d-6cda-45bf-9a42-62638fbf413c None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:48.935353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 671/1356] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:48.941201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7e2ba-86c4-4b99-9d17-7d5ca5b65712 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:48.942078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7e2ba-86c4-4b99-9d17-7d5ca5b65712 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:48.942556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7e2ba-86c4-4b99-9d17-7d5ca5b65712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:49.018893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-60e7e2ba-86c4-4b99-9d17-7d5ca5b65712 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:49.020148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-60e7e2ba-86c4-4b99-9d17-7d5ca5b65712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:49.124779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 686/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:49.179890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee068d8a-8e79-436f-93f9-02a8e067b70a None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:49.180574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee068d8a-8e79-436f-93f9-02a8e067b70a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:49.181079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee068d8a-8e79-436f-93f9-02a8e067b70a None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:49.183094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 672/1358] 10.222.0.30 () {54 vars in 962 bytes} [Fri Aug 5 05:05:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:49.191389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21719f-9a48-47f7-8ee4-bb4532731978 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:49.192755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21719f-9a48-47f7-8ee4-bb4532731978 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:49.193483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21719f-9a48-47f7-8ee4-bb4532731978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:49.215246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:49.277960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-af21719f-9a48-47f7-8ee4-bb4532731978 None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:49.279138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af21719f-9a48-47f7-8ee4-bb4532731978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:49.400129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 687/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Fri Aug 5 05:05:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:05:49.408243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee068d8a-8e79-436f-93f9-02a8e067b70a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:49.409797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ee068d8a-8e79-436f-93f9-02a8e067b70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:49.578342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee068d8a-8e79-436f-93f9-02a8e067b70a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:49.656060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:49.699108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:49.699742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:49.700111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:49.700400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:49.702027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:49.797657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:49.799632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78923736-83cd-4c65-9e73-731fea5a0bdc service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:49.826144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 673/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:52.016930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754f4bf-458e-462f-a21c-06e380dafc51 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:52.017427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754f4bf-458e-462f-a21c-06e380dafc51 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:52.017427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754f4bf-458e-462f-a21c-06e380dafc51 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:52.024245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2a30a-4aef-4c23-9317-9d2927fef261 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:52.024761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2a30a-4aef-4c23-9317-9d2927fef261 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:52.024761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2a30a-4aef-4c23-9317-9d2927fef261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:52.029111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 688/1361] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:52.093317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d5e2a30a-4aef-4c23-9317-9d2927fef261 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:52.094422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d5e2a30a-4aef-4c23-9317-9d2927fef261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:52.133509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:52.196096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 674/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:05:52.240020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754f4bf-458e-462f-a21c-06e380dafc51 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:52.241651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6754f4bf-458e-462f-a21c-06e380dafc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:52.315405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:52.406942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754f4bf-458e-462f-a21c-06e380dafc51 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:52.555038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:52.555852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:52.556097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:52.556372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:52.558506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:52.583338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:52.658369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:52.660022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab2c3c-c3ca-429f-b73f-1fe22630498c service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:52.679113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 689/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:55.038812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985bb13-9f07-413d-89cc-eba9f91f97e5 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:55.039343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985bb13-9f07-413d-89cc-eba9f91f97e5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:55.039343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985bb13-9f07-413d-89cc-eba9f91f97e5 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:55.040796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 675/1364] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:55.046743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e30d39-c0f8-438a-b050-0628f6fd143f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:55.047179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e30d39-c0f8-438a-b050-0628f6fd143f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:55.047324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e30d39-c0f8-438a-b050-0628f6fd143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:55.126896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-31e30d39-c0f8-438a-b050-0628f6fd143f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:55.128277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-31e30d39-c0f8-438a-b050-0628f6fd143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:55.236612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:55.247831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 690/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:55.291052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db496-cc3c-43e3-aec6-24d287c9e31f None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:55.291368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db496-cc3c-43e3-aec6-24d287c9e31f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:55.291600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db496-cc3c-43e3-aec6-24d287c9e31f None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:55.292985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 676/1366] 10.222.0.30 () {54 vars in 962 bytes} [Fri Aug 5 05:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:55.301757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f436-6044-4878-9099-17ce87043d9e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:55.302050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f436-6044-4878-9099-17ce87043d9e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:55.302331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f436-6044-4878-9099-17ce87043d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:55.397174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-50c7f436-6044-4878-9099-17ce87043d9e None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:55.398491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-50c7f436-6044-4878-9099-17ce87043d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:55.531949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 691/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Fri Aug 5 05:05:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:55.540343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db496-cc3c-43e3-aec6-24d287c9e31f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:55.541985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-435db496-cc3c-43e3-aec6-24d287c9e31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:55.709456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db496-cc3c-43e3-aec6-24d287c9e31f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:55.730393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:55.828736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:55.829427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:55.829713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:55.829991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:55.831607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:55.927072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:55.928701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8ced-951c-4ba6-b8ed-d47604625947 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:55.948767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 677/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:56.925823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ab52-0e62-4554-8f7a-ea17e7bcb3b6 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:56.926422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ab52-0e62-4554-8f7a-ea17e7bcb3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:56.926713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ab52-0e62-4554-8f7a-ea17e7bcb3b6 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:56.928702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 692/1369] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:56.934526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f701a-3053-447b-b31c-cfbd3d9445a9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:56.935048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f701a-3053-447b-b31c-cfbd3d9445a9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:56.935131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f701a-3053-447b-b31c-cfbd3d9445a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:56.973725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:57.023074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6d0f701a-3053-447b-b31c-cfbd3d9445a9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:57.024406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6d0f701a-3053-447b-b31c-cfbd3d9445a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:57.135591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 678/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:57.494770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b0007-71db-4ba2-bc23-e7d64a18e040 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:57.494770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b0007-71db-4ba2-bc23-e7d64a18e040 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:57.494770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b0007-71db-4ba2-bc23-e7d64a18e040 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:57.494770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 693/1371] 10.222.0.30 () {54 vars in 964 bytes} [Fri Aug 5 05:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:57.502650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae0738-b908-43c5-a993-228aa06c2091 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:57.502650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae0738-b908-43c5-a993-228aa06c2091 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:57.502650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae0738-b908-43c5-a993-228aa06c2091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:57.582458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-25ae0738-b908-43c5-a993-228aa06c2091 None None] MFA Rules not processed for user `ea0805461cd3492fa5e2ebbea407e351`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:57.583676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-25ae0738-b908-43c5-a993-228aa06c2091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:57.632503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:57.664249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 679/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Fri Aug 5 05:05:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:05:57.671996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b0007-71db-4ba2-bc23-e7d64a18e040 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:57.673907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d32b0007-71db-4ba2-bc23-e7d64a18e040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:57.726362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:57.781826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b0007-71db-4ba2-bc23-e7d64a18e040 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:57.856545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:57.857176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:57.857457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:57.857732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:57.859344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:57.965304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:57.979391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:57.980972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4a505-ed03-48ec-852f-9634d4e0ea36 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:58.002089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9453eb6-88e9-4f72-a499-6057c4365f13 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:58.002631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9453eb6-88e9-4f72-a499-6057c4365f13 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:58.003437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9453eb6-88e9-4f72-a499-6057c4365f13 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:58.004082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 694/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:05:58.006512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 680/1374] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:05:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:05:58.012152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbe5ad-1430-4f6c-8604-8fbe9c9c2e9a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:58.012152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbe5ad-1430-4f6c-8604-8fbe9c9c2e9a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:58.012152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbe5ad-1430-4f6c-8604-8fbe9c9c2e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:58.081693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-93dbe5ad-1430-4f6c-8604-8fbe9c9c2e9a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:05:58.082910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-93dbe5ad-1430-4f6c-8604-8fbe9c9c2e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:58.208389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 695/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:05:58.240952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9453eb6-88e9-4f72-a499-6057c4365f13 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:05:58.240952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d9453eb6-88e9-4f72-a499-6057c4365f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:58.410453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9453eb6-88e9-4f72-a499-6057c4365f13 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:05:58.461519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:05:58.523232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:05:58.523796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:05:58.524082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:05:58.524349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:05:58.525980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:05:58.618185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:05:58.619802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525acfe-2ca3-4f91-afdc-ca9de905616f service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:05:58.638277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 681/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:00.666555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeab55-0e1a-449b-a574-13bbcb145a77 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:00.667068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeab55-0e1a-449b-a574-13bbcb145a77 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:00.667193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeab55-0e1a-449b-a574-13bbcb145a77 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:00.668507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 696/1377] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:00.674224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80c30-2144-459f-b9be-6529f7cc56d4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:00.674621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80c30-2144-459f-b9be-6529f7cc56d4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:00.674766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80c30-2144-459f-b9be-6529f7cc56d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:00.741966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1bc80c30-2144-459f-b9be-6529f7cc56d4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:00.743182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1bc80c30-2144-459f-b9be-6529f7cc56d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:00.750499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:00.845708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 682/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:00.890523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeab55-0e1a-449b-a574-13bbcb145a77 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:00.892365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-18eeab55-0e1a-449b-a574-13bbcb145a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:00.969814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:01.084386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeab55-0e1a-449b-a574-13bbcb145a77 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:01.196070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:01.196691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:01.196988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:01.197243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:01.198850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:01.217047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:01.293443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:01.294967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b82e5-00b5-4050-aa04-11e12f68bafc service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:01.313620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 697/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:03.360636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a45e9-b7ce-48c5-b8e1-8e69fe0a793c None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:03.361280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a45e9-b7ce-48c5-b8e1-8e69fe0a793c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:03.361280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a45e9-b7ce-48c5-b8e1-8e69fe0a793c None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:03.362516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 683/1380] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:03.368060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092402-9b4b-4fe5-b521-04e2ef36418d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:03.368543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092402-9b4b-4fe5-b521-04e2ef36418d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:03.368813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092402-9b4b-4fe5-b521-04e2ef36418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:03.442075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f0092402-9b4b-4fe5-b521-04e2ef36418d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:03.443593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f0092402-9b4b-4fe5-b521-04e2ef36418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:03.551161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:03.559907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 698/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:03.601079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a45e9-b7ce-48c5-b8e1-8e69fe0a793c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:03.602594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2e7a45e9-b7ce-48c5-b8e1-8e69fe0a793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:03.742120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:03.758370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a45e9-b7ce-48c5-b8e1-8e69fe0a793c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:03.878273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:03.878811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:03.879090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:03.879361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:03.881014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:03.973305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:03.974891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02784f-0b76-4b0d-bfe3-c8c2effd8b34 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:03.985889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:04.006412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 684/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:06.044447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0263-3d80-4ffe-8587-d81b5014dcb8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:06.045034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0263-3d80-4ffe-8587-d81b5014dcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:06.045034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0263-3d80-4ffe-8587-d81b5014dcb8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:06.046451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 699/1383] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:06.052845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c6ad-7b3b-47e4-8d44-0adb4c5b4d18 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:06.053496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c6ad-7b3b-47e4-8d44-0adb4c5b4d18 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:06.053496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c6ad-7b3b-47e4-8d44-0adb4c5b4d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:06.135472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2fb9c6ad-7b3b-47e4-8d44-0adb4c5b4d18 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:06.136820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2fb9c6ad-7b3b-47e4-8d44-0adb4c5b4d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:06.243365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 685/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:06.289789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0263-3d80-4ffe-8587-d81b5014dcb8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:06.291393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d4cd0263-3d80-4ffe-8587-d81b5014dcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:06.460322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0263-3d80-4ffe-8587-d81b5014dcb8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:06.560368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:06.625085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:06.625830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:06.626095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:06.626372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:06.628133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:06.722725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:06.724479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377ad8-66ef-421b-9be4-dea71ef0ce0e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:06.743155 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 700/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:08.778443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb5cc-600f-48f9-9c90-079875318ed3 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:08.779040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb5cc-600f-48f9-9c90-079875318ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:08.779040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb5cc-600f-48f9-9c90-079875318ed3 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:08.780578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 686/1386] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:08.786867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16a8cc-b201-4b8a-a03b-4c8dea893292 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:08.787341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16a8cc-b201-4b8a-a03b-4c8dea893292 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:08.787728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16a8cc-b201-4b8a-a03b-4c8dea893292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:08.861374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2b16a8cc-b201-4b8a-a03b-4c8dea893292 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:08.862675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2b16a8cc-b201-4b8a-a03b-4c8dea893292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:08.896635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:08.971370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 701/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:09.018345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eb5cc-600f-48f9-9c90-079875318ed3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:09.020671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-159eb5cc-600f-48f9-9c90-079875318ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:09.125956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:09.204797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eb5cc-600f-48f9-9c90-079875318ed3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:09.323452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:09.324070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:09.324401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:09.324600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:09.326354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:09.369741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:09.424168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:09.425843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1fb7-9e0c-4c1c-9d15-41c60224d78f service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:09.445099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 687/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:11.852343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33772236-ada3-42de-8be2-cf0c16c9cfd7 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:11.853067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33772236-ada3-42de-8be2-cf0c16c9cfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:11.853067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33772236-ada3-42de-8be2-cf0c16c9cfd7 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:11.854309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 702/1389] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:11.859704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf49e0-589e-443c-af82-4243e3a58f78 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:11.860174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf49e0-589e-443c-af82-4243e3a58f78 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:11.860431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf49e0-589e-443c-af82-4243e3a58f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:11.926774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fbaf49e0-589e-443c-af82-4243e3a58f78 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:11.928003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fbaf49e0-589e-443c-af82-4243e3a58f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:12.023540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 688/1390] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:12.068618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33772236-ada3-42de-8be2-cf0c16c9cfd7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:12.070120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-33772236-ada3-42de-8be2-cf0c16c9cfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:12.216653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33772236-ada3-42de-8be2-cf0c16c9cfd7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:12.233172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:12.334599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:12.335230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:12.335508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:12.335806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:12.337413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:12.429987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:12.431614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed9f5-e693-466e-b8d4-f46a29b32c29 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:12.449969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 703/1391] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:12.935634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73a72-fbee-4110-b032-bc8d2b7f8ea1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:12.936245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73a72-fbee-4110-b032-bc8d2b7f8ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:12.936245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73a72-fbee-4110-b032-bc8d2b7f8ea1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:12.937631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 689/1392] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:12.942993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbec4d-6bc9-4d75-a7aa-e128547c6bfd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:12.943459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbec4d-6bc9-4d75-a7aa-e128547c6bfd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:12.943562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbec4d-6bc9-4d75-a7aa-e128547c6bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:12.969431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:13.015356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b1bbec4d-6bc9-4d75-a7aa-e128547c6bfd None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:13.016617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b1bbec4d-6bc9-4d75-a7aa-e128547c6bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:13.241091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 704/1393] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:13.296532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14070c-0e88-4b9f-88da-01c996653ae7 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:13.296819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14070c-0e88-4b9f-88da-01c996653ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:13.297061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14070c-0e88-4b9f-88da-01c996653ae7 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:13.298878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 690/1394] 10.222.0.30 () {54 vars in 966 bytes} [Fri Aug 5 05:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:13.303869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb11c-1062-4fc6-8875-7e83c27a4927 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:13.304153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb11c-1062-4fc6-8875-7e83c27a4927 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:13.304420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb11c-1062-4fc6-8875-7e83c27a4927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:13.378095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b51fb11c-1062-4fc6-8875-7e83c27a4927 None None] MFA Rules not processed for user `a88bb908a0104cc589b19095091a8a53`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:13.379292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b51fb11c-1062-4fc6-8875-7e83c27a4927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:13.420925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:13.457807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 705/1395] 10.222.0.30 () {58 vars in 1063 bytes} [Fri Aug 5 05:06:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:13.465607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14070c-0e88-4b9f-88da-01c996653ae7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:13.467109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-af14070c-0e88-4b9f-88da-01c996653ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:13.481040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:13.574271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14070c-0e88-4b9f-88da-01c996653ae7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:13.650342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:13.650967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:13.651247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:13.651517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:13.653168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:13.720408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:13.762516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:13.764021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081498c-0f63-432e-8a3c-5e25ce9e2b84 service glance] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:13.782507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 691/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:14.466776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31977d1-3ec9-4612-9051-48be70b18883 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:14.467390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31977d1-3ec9-4612-9051-48be70b18883 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:14.467390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31977d1-3ec9-4612-9051-48be70b18883 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:14.468817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 706/1397] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:14.475023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:14.475462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:14.475641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:14.519831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81f3d2-592f-4eab-91bb-c10471c2ac62 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:14.520123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81f3d2-592f-4eab-91bb-c10471c2ac62 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:14.520423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81f3d2-592f-4eab-91bb-c10471c2ac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:14.545743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:14.546956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:14.588455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-df81f3d2-592f-4eab-91bb-c10471c2ac62 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:14.589780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df81f3d2-592f-4eab-91bb-c10471c2ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:14.645190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 692/1398] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:14.677506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 707/1399] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:06:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:06:14.690771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:14.692032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:14.723531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:14.723963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36366ad-9d27-422f-91fd-5244367f0db0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:14.724253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36366ad-9d27-422f-91fd-5244367f0db0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:14.724533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36366ad-9d27-422f-91fd-5244367f0db0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:14.725913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 708/1400] 10.222.0.30 () {54 vars in 943 bytes} [Fri Aug 5 05:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:14.731909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:14.732182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:14.732459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:14.804443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:14.812464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] MFA Rules not processed for user `127c0f72286541159eac49c8f958f816`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:14.813109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13973d1b-0cc7-4ee2-847a-6a7e349ce6f0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:14.813715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:14.921954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 709/1401] 10.222.0.30 () {58 vars in 1040 bytes} [Fri Aug 5 05:06:14 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:06:14.929951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:14.930669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:14.930859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:14.931129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:14.931740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:14.932795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:14.941053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092d28e-3c7d-4da8-837a-7a52ac3cc9c5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:14.950808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:15.023015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:15.024664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452880b-c94f-4f99-99ec-9a1d44461377 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:15.042483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 693/1402] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:15.049988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:15.050618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:15.050872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:15.051137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:15.052859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:15.055403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:15.144694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:15.146750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7a274-fed9-4ad3-a168-399a874f4282 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:15.157900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 710/1403] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:06:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 05:06:17.130453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc53ce-81ed-45ed-b825-dcd5e00005d8 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:17.131024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc53ce-81ed-45ed-b825-dcd5e00005d8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:17.131024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc53ce-81ed-45ed-b825-dcd5e00005d8 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:17.135997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 694/1404] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:17.137585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98734566-28d7-4904-b8f3-2f2caa9a7134 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:17.138005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98734566-28d7-4904-b8f3-2f2caa9a7134 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:17.138188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98734566-28d7-4904-b8f3-2f2caa9a7134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:17.217687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-98734566-28d7-4904-b8f3-2f2caa9a7134 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:17.217687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98734566-28d7-4904-b8f3-2f2caa9a7134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:17.333245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:17.343819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 711/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:17.385222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc53ce-81ed-45ed-b825-dcd5e00005d8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:17.386673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-48fc53ce-81ed-45ed-b825-dcd5e00005d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:17.550772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:17.553506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc53ce-81ed-45ed-b825-dcd5e00005d8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:17.679570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:17.680232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:17.680508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:17.680796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:17.682406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:17.779701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:17.781800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e70bc6-cfd1-4829-ba51-6273c6dbee7e service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:17.804378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 695/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:19.819892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe4e95-48bc-4ee9-b6b9-d3fa3171516a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:19.820444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe4e95-48bc-4ee9-b6b9-d3fa3171516a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:19.820444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe4e95-48bc-4ee9-b6b9-d3fa3171516a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:19.821706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 712/1407] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:19.826870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d081dfe-d303-4be3-9363-ed37efcad0ce None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:19.827323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d081dfe-d303-4be3-9363-ed37efcad0ce None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:19.827448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d081dfe-d303-4be3-9363-ed37efcad0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:19.832069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:19.903089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7d081dfe-d303-4be3-9363-ed37efcad0ce None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:19.904352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7d081dfe-d303-4be3-9363-ed37efcad0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:20.013510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 696/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:06:20.055031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe4e95-48bc-4ee9-b6b9-d3fa3171516a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:20.056602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-89fe4e95-48bc-4ee9-b6b9-d3fa3171516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:20.220341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe4e95-48bc-4ee9-b6b9-d3fa3171516a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:20.324723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:20.411387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:20.412061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:20.412342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:20.412615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:20.414245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:20.507296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:20.509369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb14eb-2e3a-416a-9617-6488e78c340e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:20.540047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 713/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:30.008884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=64 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:30.527725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=64 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:37.238156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9d202-2272-4945-b309-00b491c35132 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:37.238772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9d202-2272-4945-b309-00b491c35132 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:37.238772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9d202-2272-4945-b309-00b491c35132 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:37.240769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 697/1410] 10.222.0.30 () {54 vars in 918 bytes} [Fri Aug 5 05:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:37.245952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cf53e-109b-40b2-93bc-914d12fbb75f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:37.246381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cf53e-109b-40b2-93bc-914d12fbb75f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:37.246491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cf53e-109b-40b2-93bc-914d12fbb75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:37.335871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eb9cf53e-109b-40b2-93bc-914d12fbb75f None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:37.337120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eb9cf53e-109b-40b2-93bc-914d12fbb75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:37.476025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 714/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Fri Aug 5 05:06:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:37.499788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e7b2-3131-4c45-bf2c-65eee41bbdf1 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:37.500072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e7b2-3131-4c45-bf2c-65eee41bbdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:37.500349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e7b2-3131-4c45-bf2c-65eee41bbdf1 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:37.501747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 698/1412] 10.222.0.30 () {54 vars in 918 bytes} [Fri Aug 5 05:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:37.506663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17661228-027b-4c83-81e3-cb4aa6e4159b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:37.506951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17661228-027b-4c83-81e3-cb4aa6e4159b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:37.507223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17661228-027b-4c83-81e3-cb4aa6e4159b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:37.581726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:37.584778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-17661228-027b-4c83-81e3-cb4aa6e4159b None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:37.586745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-17661228-027b-4c83-81e3-cb4aa6e4159b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:37.704860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 715/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Fri Aug 5 05:06:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:40.162299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed4c9-6f9f-4ca3-b468-02eab07309e8 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:40.163143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed4c9-6f9f-4ca3-b468-02eab07309e8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:40.163143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed4c9-6f9f-4ca3-b468-02eab07309e8 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:40.165394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:40.173847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0e627-bc29-435f-888d-228149c02e97 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:40.174293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0e627-bc29-435f-888d-228149c02e97 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:40.174399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0e627-bc29-435f-888d-228149c02e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:40.242408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-28b0e627-bc29-435f-888d-228149c02e97 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:40.243624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28b0e627-bc29-435f-888d-228149c02e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:40.308755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:40.352876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:06:40.396445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ed4c9-6f9f-4ca3-b468-02eab07309e8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:40.398877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-753ed4c9-6f9f-4ca3-b468-02eab07309e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:40.559894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ed4c9-6f9f-4ca3-b468-02eab07309e8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:40.636027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:40.678205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:40.678846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:40.679120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:40.679387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:40.681050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:40.772929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:40.774529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b7ebd-58a6-423e-8646-b57a69956b7f service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:40.793164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 700/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:41.684626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39837-aaba-4bdc-be29-233021d09c14 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:41.685593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39837-aaba-4bdc-be29-233021d09c14 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:41.685593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39837-aaba-4bdc-be29-233021d09c14 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:41.687672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:41.693934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fdfd-b948-45c4-a32f-93a8a5a66b73 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:41.694641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fdfd-b948-45c4-a32f-93a8a5a66b73 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:41.694818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fdfd-b948-45c4-a32f-93a8a5a66b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:41.779358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d447fdfd-b948-45c4-a32f-93a8a5a66b73 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:41.781009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d447fdfd-b948-45c4-a32f-93a8a5a66b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:41.809812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:41.889576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:42.042079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39837-aaba-4bdc-be29-233021d09c14 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:42.043634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-65e39837-aaba-4bdc-be29-233021d09c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:42.194849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39837-aaba-4bdc-be29-233021d09c14 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:42.234990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:42.310021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:42.310569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:42.310847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:42.311114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:42.312808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:42.410622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:42.412314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e7795-1d6a-41fc-9f0a-18c0ddcecf7e service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:42.432058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 718/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:43.314135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2366b-4225-438a-9c13-d57f1c489d40 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:43.314963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2366b-4225-438a-9c13-d57f1c489d40 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:43.314963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2366b-4225-438a-9c13-d57f1c489d40 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:43.316345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:43.322533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105a0ab-b560-48b5-8f7a-56fdc5fe87ba None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:43.322967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105a0ab-b560-48b5-8f7a-56fdc5fe87ba None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:43.323115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105a0ab-b560-48b5-8f7a-56fdc5fe87ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:43.381101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:43.404986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1105a0ab-b560-48b5-8f7a-56fdc5fe87ba None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:43.406199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1105a0ab-b560-48b5-8f7a-56fdc5fe87ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:43.512340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:43.670463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2366b-4225-438a-9c13-d57f1c489d40 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:43.672145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-95f2366b-4225-438a-9c13-d57f1c489d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:43.840417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2366b-4225-438a-9c13-d57f1c489d40 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:43.853185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:43.978311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:43.978846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:43.979123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:43.979420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:43.981096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:44.079652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:44.081368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da434f54-fc19-4055-981f-7245edf7ef9d service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:44.102725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 703/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:44.962977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741af5f0-2305-4ca8-b9c6-bd7b536d606d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:44.963486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741af5f0-2305-4ca8-b9c6-bd7b536d606d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:44.963592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741af5f0-2305-4ca8-b9c6-bd7b536d606d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:44.965742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:44.971955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcba5e-f0e9-461e-8ca9-eb095a502025 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:44.972390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcba5e-f0e9-461e-8ca9-eb095a502025 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:44.972534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcba5e-f0e9-461e-8ca9-eb095a502025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:44.998487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:45.060499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-04dcba5e-f0e9-461e-8ca9-eb095a502025 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:45.061718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-04dcba5e-f0e9-461e-8ca9-eb095a502025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:45.160041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:45.314205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741af5f0-2305-4ca8-b9c6-bd7b536d606d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:45.316131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-741af5f0-2305-4ca8-b9c6-bd7b536d606d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:45.436622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:45.472343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741af5f0-2305-4ca8-b9c6-bd7b536d606d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:45.587300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:45.587941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:45.588217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:45.588487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:45.590081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:45.668954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:45.683705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:45.685362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443da50c-936b-4c6e-8c07-60b056930078 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:45.703211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 721/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:46.550933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c1ac-1577-4eab-8a9e-f194d994b1bd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:46.551538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c1ac-1577-4eab-8a9e-f194d994b1bd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:46.551538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c1ac-1577-4eab-8a9e-f194d994b1bd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:46.552878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:06:46.558953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100467e-948d-4d4b-b648-bd86d1d96d13 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:46.559664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100467e-948d-4d4b-b648-bd86d1d96d13 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:46.559868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100467e-948d-4d4b-b648-bd86d1d96d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:46.647363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4100467e-948d-4d4b-b648-bd86d1d96d13 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:46.648610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4100467e-948d-4d4b-b648-bd86d1d96d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:46.752576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:46.902226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c1ac-1577-4eab-8a9e-f194d994b1bd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:46.904422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1b45c1ac-1577-4eab-8a9e-f194d994b1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:46.998135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:47.061725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c1ac-1577-4eab-8a9e-f194d994b1bd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:47.193730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:47.194362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:47.194638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:47.194929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:47.196584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:47.245396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:47.288089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:47.289700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b77ce-42d9-4274-9ee8-4afc541832d9 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:47.308592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 706/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:48.101403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732e91f-f48a-4ebb-819f-a99827d10674 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:48.102292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732e91f-f48a-4ebb-819f-a99827d10674 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:48.102292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732e91f-f48a-4ebb-819f-a99827d10674 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:48.104522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:48.112621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638d6c9-d849-4752-94e9-ed057f165839 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:48.113043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638d6c9-d849-4752-94e9-ed057f165839 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:48.113212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638d6c9-d849-4752-94e9-ed057f165839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:48.190604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e638d6c9-d849-4752-94e9-ed057f165839 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:48.191826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e638d6c9-d849-4752-94e9-ed057f165839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:48.294384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:48.445501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732e91f-f48a-4ebb-819f-a99827d10674 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:48.447006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1732e91f-f48a-4ebb-819f-a99827d10674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:48.498213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:48.607039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732e91f-f48a-4ebb-819f-a99827d10674 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:48.725505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:48.726121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:48.726399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:48.726668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:48.728320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:48.735380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:48.828117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:48.830566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dba9c-f6ef-4925-8765-c26d1f1fec0b service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:48.849465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 724/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:49.642073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69da2aa-dc93-4452-9e5a-f352ff5ce314 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:49.642686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69da2aa-dc93-4452-9e5a-f352ff5ce314 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:49.642686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69da2aa-dc93-4452-9e5a-f352ff5ce314 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:49.644076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:49.650088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08033-8bcd-4f8f-b15b-b3dc0db319a7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:49.650550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08033-8bcd-4f8f-b15b-b3dc0db319a7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:49.650645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08033-8bcd-4f8f-b15b-b3dc0db319a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:49.740085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-00a08033-8bcd-4f8f-b15b-b3dc0db319a7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:49.741295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-00a08033-8bcd-4f8f-b15b-b3dc0db319a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:49.818910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:49.848553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:49.998931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69da2aa-dc93-4452-9e5a-f352ff5ce314 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:50.000515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c69da2aa-dc93-4452-9e5a-f352ff5ce314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:50.018496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:50.162900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69da2aa-dc93-4452-9e5a-f352ff5ce314 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:50.259551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:50.286892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:50.287518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:50.287809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:50.288095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:50.289722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:50.382226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:50.383882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f285-b03a-4afe-9dd1-29a98b301a16 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:50.402211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 709/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:51.203306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ead065-f524-4dd9-9b60-cd241d89b97e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:51.203892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ead065-f524-4dd9-9b60-cd241d89b97e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:51.203892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ead065-f524-4dd9-9b60-cd241d89b97e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:51.205811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:51.210978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c66e-d059-48cb-a7d7-19e1af22399c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:51.211433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c66e-d059-48cb-a7d7-19e1af22399c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:51.211537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9c66e-d059-48cb-a7d7-19e1af22399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:51.291134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-95f9c66e-d059-48cb-a7d7-19e1af22399c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:51.292383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-95f9c66e-d059-48cb-a7d7-19e1af22399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:51.337857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:51.394189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:51.548918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ead065-f524-4dd9-9b60-cd241d89b97e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:51.551190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c4ead065-f524-4dd9-9b60-cd241d89b97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:51.706652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ead065-f524-4dd9-9b60-cd241d89b97e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:51.771881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:51.832505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:51.833024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:51.833333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:51.833572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:51.835201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:51.926747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:51.928365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9231f70-813f-4001-8120-0a63f38ebce8 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:51.948303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 727/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:52.752475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052391-ea98-4ecb-9d2c-d887e5e14b33 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:52.753351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052391-ea98-4ecb-9d2c-d887e5e14b33 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:52.753351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052391-ea98-4ecb-9d2c-d887e5e14b33 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:52.755483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:06:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:52.763965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1cdc-2bc5-480b-968b-ae97e61f67ac None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:52.764396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1cdc-2bc5-480b-968b-ae97e61f67ac None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:52.764705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1cdc-2bc5-480b-968b-ae97e61f67ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:52.849079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-56ee1cdc-2bc5-480b-968b-ae97e61f67ac None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:52.850285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56ee1cdc-2bc5-480b-968b-ae97e61f67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:52.852360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:52.966145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:53.117258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91052391-ea98-4ecb-9d2c-d887e5e14b33 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:53.119416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-91052391-ea98-4ecb-9d2c-d887e5e14b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:53.300919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91052391-ea98-4ecb-9d2c-d887e5e14b33 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:53.333414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:53.426220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:53.426853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:53.427129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:53.427401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:53.429079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:53.533404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:53.535075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e18835-6ee0-4328-930b-e82467c8481e service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:53.553626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 712/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:55.470711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a431-e6e5-42e7-be77-3d2f9129e0f0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:55.471263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a431-e6e5-42e7-be77-3d2f9129e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:55.471263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a431-e6e5-42e7-be77-3d2f9129e0f0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:55.472669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:55.478516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2527ca8-86e0-4640-ac46-c8bacc099cb7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:55.478972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2527ca8-86e0-4640-ac46-c8bacc099cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:55.479128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2527ca8-86e0-4640-ac46-c8bacc099cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:55.520874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:55.557092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f2527ca8-86e0-4640-ac46-c8bacc099cb7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:55.558228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f2527ca8-86e0-4640-ac46-c8bacc099cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:55.675298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:06:55.717858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a431-e6e5-42e7-be77-3d2f9129e0f0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:55.720295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9b26a431-e6e5-42e7-be77-3d2f9129e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:55.830906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a431-e6e5-42e7-be77-3d2f9129e0f0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:55.864044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:55.906849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:55.907482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:55.907787 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:55.908067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:55.909742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:56.027644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:56.029150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bacca-5654-4ab8-9691-f0a370b5508d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:56.046759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 730/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:06:58.183112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb6ed-3032-4d51-939f-3602abdfe9be None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:58.184066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb6ed-3032-4d51-939f-3602abdfe9be None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:58.184066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb6ed-3032-4d51-939f-3602abdfe9be None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:58.186182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:06:58.192457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8d0be-fdcf-4d58-b7d3-bb4c8a238c23 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:58.192932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8d0be-fdcf-4d58-b7d3-bb4c8a238c23 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:58.193056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8d0be-fdcf-4d58-b7d3-bb4c8a238c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:58.269719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c2f8d0be-fdcf-4d58-b7d3-bb4c8a238c23 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:06:58.270889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c2f8d0be-fdcf-4d58-b7d3-bb4c8a238c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:58.284697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:58.383557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:06:58.422101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb6ed-3032-4d51-939f-3602abdfe9be None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:06:58.424641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-68afb6ed-3032-4d51-939f-3602abdfe9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:58.532075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb6ed-3032-4d51-939f-3602abdfe9be None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:06:58.544319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:06:58.620435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:06:58.621057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:06:58.621336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:06:58.621605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:06:58.623216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:06:58.741600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:06:58.743642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565c88-a872-4ebf-b81c-ceae621f56fb service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:06:58.768011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 715/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:01.066721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c6fce-959d-4f1f-80a1-6b24ed026913 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:01.067540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c6fce-959d-4f1f-80a1-6b24ed026913 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:01.067697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c6fce-959d-4f1f-80a1-6b24ed026913 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:01.069756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:01.077731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c6d66-a922-49b4-a212-80f76fdacc03 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:01.078206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c6d66-a922-49b4-a212-80f76fdacc03 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:01.078307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c6d66-a922-49b4-a212-80f76fdacc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:01.130109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:01.147880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a60c6d66-a922-49b4-a212-80f76fdacc03 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:01.149085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a60c6d66-a922-49b4-a212-80f76fdacc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:01.249052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:01.288134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c6fce-959d-4f1f-80a1-6b24ed026913 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:01.289688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-aa3c6fce-959d-4f1f-80a1-6b24ed026913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:01.403503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c6fce-959d-4f1f-80a1-6b24ed026913 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:01.445607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:01.481354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:01.481983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:01.482298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:01.482576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:01.484339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:01.620579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:01.621940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfff995-ae3d-4b3b-a1e5-d9caff317069 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:01.642151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 733/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:03.758620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42753a58-ca41-4de5-8f14-0e8875deed4b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:03.759191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42753a58-ca41-4de5-8f14-0e8875deed4b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:03.759191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42753a58-ca41-4de5-8f14-0e8875deed4b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:03.760584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:03.767055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072d57-4f7e-46f4-a8af-489a8ae01395 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:03.767842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072d57-4f7e-46f4-a8af-489a8ae01395 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:03.767974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072d57-4f7e-46f4-a8af-489a8ae01395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:03.839850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c7072d57-4f7e-46f4-a8af-489a8ae01395 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:03.841073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7072d57-4f7e-46f4-a8af-489a8ae01395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:03.862710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:03.943921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:03.984114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42753a58-ca41-4de5-8f14-0e8875deed4b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:03.986477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-42753a58-ca41-4de5-8f14-0e8875deed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:04.098107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42753a58-ca41-4de5-8f14-0e8875deed4b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:04.120949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:04.177093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:04.177710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:04.177987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:04.178260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:04.179903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:04.297007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:04.298814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e2373-e7c6-41f9-b862-630b8a1d8017 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:04.318075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 718/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:06.358249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e659d8d-5812-4ebc-b37f-9e97e7315f1d None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:06.358804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e659d8d-5812-4ebc-b37f-9e97e7315f1d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:06.358804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e659d8d-5812-4ebc-b37f-9e97e7315f1d None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:06.360134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:06.366348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f714753-5ec7-4496-8f61-1272937329e4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:06.367060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f714753-5ec7-4496-8f61-1272937329e4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:06.367278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f714753-5ec7-4496-8f61-1272937329e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:06.435011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:06.437752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4f714753-5ec7-4496-8f61-1272937329e4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:06.439812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4f714753-5ec7-4496-8f61-1272937329e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:06.538069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:06.580533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e659d8d-5812-4ebc-b37f-9e97e7315f1d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:06.582057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e659d8d-5812-4ebc-b37f-9e97e7315f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:06.690272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e659d8d-5812-4ebc-b37f-9e97e7315f1d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:06.742104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:06.769688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:06.770317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:06.770597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:06.770870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:06.772531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:06.897999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:06.899451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1dcf-a41e-4c8a-ac0e-799a9c562a48 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:06.917776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 736/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:08.955928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86df89-df03-42bf-9f27-705b6d8a8955 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:08.956427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86df89-df03-42bf-9f27-705b6d8a8955 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:08.956548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86df89-df03-42bf-9f27-705b6d8a8955 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:08.957891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:08.963391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4bc48-d4db-431c-bd0b-d84817ccfc65 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:08.963842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4bc48-d4db-431c-bd0b-d84817ccfc65 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:08.963979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4bc48-d4db-431c-bd0b-d84817ccfc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:09.032138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-95b4bc48-d4db-431c-bd0b-d84817ccfc65 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:09.033392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-95b4bc48-d4db-431c-bd0b-d84817ccfc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:09.063630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:09.130032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:09.169063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86df89-df03-42bf-9f27-705b6d8a8955 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:09.171101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ec86df89-df03-42bf-9f27-705b6d8a8955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:09.280227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86df89-df03-42bf-9f27-705b6d8a8955 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:09.310404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:09.356206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:09.356832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:09.357116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:09.357393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:09.359043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:09.473164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:09.474626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef7a43-86d2-4f8f-9ce3-16a854f39ce8 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:09.493052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 721/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:11.552699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268972d2-8133-4021-ab29-942133c74d47 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:11.553566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268972d2-8133-4021-ab29-942133c74d47 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:11.553566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268972d2-8133-4021-ab29-942133c74d47 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:11.555699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:11.562272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ef57-55c0-4926-8610-fcd16a10083c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:11.563014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ef57-55c0-4926-8610-fcd16a10083c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:11.563141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ef57-55c0-4926-8610-fcd16a10083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:11.632546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f5c2ef57-55c0-4926-8610-fcd16a10083c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:11.633756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f5c2ef57-55c0-4926-8610-fcd16a10083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:11.642397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:11.742777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:11.785727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268972d2-8133-4021-ab29-942133c74d47 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:11.788121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-268972d2-8133-4021-ab29-942133c74d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:11.908544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268972d2-8133-4021-ab29-942133c74d47 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:11.964850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:11.984603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:11.985219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:11.985495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:11.985764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:11.987406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:12.105511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:12.106976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e1d72-f053-4628-a5b6-06dd5cc4e619 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:12.125592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 739/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:14.143996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84323012-7c5f-470d-b5f4-148d4c1bdb47 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:14.144821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84323012-7c5f-470d-b5f4-148d4c1bdb47 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:14.144821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84323012-7c5f-470d-b5f4-148d4c1bdb47 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:14.146997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:14.153302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3cf0-9120-407c-b397-306162711a3a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:14.153998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3cf0-9120-407c-b397-306162711a3a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:14.154203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3cf0-9120-407c-b397-306162711a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:14.233250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bb6f3cf0-9120-407c-b397-306162711a3a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:14.234814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bb6f3cf0-9120-407c-b397-306162711a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:14.276143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:14.336237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:14.376537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84323012-7c5f-470d-b5f4-148d4c1bdb47 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:14.378163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84323012-7c5f-470d-b5f4-148d4c1bdb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:14.483061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84323012-7c5f-470d-b5f4-148d4c1bdb47 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:14.522193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:14.563030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:14.563574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:14.563878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:14.564170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:14.565816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:14.682286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:14.683713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62979d22-b9c5-4586-858c-dcb1bb28c589 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:14.702735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 724/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:07:16.710396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514aca3-50eb-437a-b25d-0388b7865810 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:16.711347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514aca3-50eb-437a-b25d-0388b7865810 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:16.711347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514aca3-50eb-437a-b25d-0388b7865810 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:16.713527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:16.720052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a19dd0-d48a-4040-ab84-ede5b0260fb7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:16.720748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a19dd0-d48a-4040-ab84-ede5b0260fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:16.720843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a19dd0-d48a-4040-ab84-ede5b0260fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:16.793969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b8a19dd0-d48a-4040-ab84-ede5b0260fb7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:16.795172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b8a19dd0-d48a-4040-ab84-ede5b0260fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:16.814854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:16.899214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:16.939894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514aca3-50eb-437a-b25d-0388b7865810 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:16.941436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0514aca3-50eb-437a-b25d-0388b7865810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:17.047936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514aca3-50eb-437a-b25d-0388b7865810 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:17.114671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:17.122523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:17.123167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:17.123450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:17.123707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:17.125398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:17.234972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:17.236601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec99c4-e12e-4d91-b398-495af876d98d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:17.255300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 742/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:19.265272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b055ab6-4537-4bc6-9521-1a2e0abcf575 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:19.266107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b055ab6-4537-4bc6-9521-1a2e0abcf575 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:19.266107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b055ab6-4537-4bc6-9521-1a2e0abcf575 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:19.268214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:07:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:19.277098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61563187-8311-43c3-924c-416fa02089f7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:19.277520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61563187-8311-43c3-924c-416fa02089f7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:19.277750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61563187-8311-43c3-924c-416fa02089f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:19.348956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-61563187-8311-43c3-924c-416fa02089f7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:19.350214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-61563187-8311-43c3-924c-416fa02089f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:19.400435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:19.455412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:19.492607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b055ab6-4537-4bc6-9521-1a2e0abcf575 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:19.494192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7b055ab6-4537-4bc6-9521-1a2e0abcf575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:19.599102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b055ab6-4537-4bc6-9521-1a2e0abcf575 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:19.659728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:19.688843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:19.689512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:19.689793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:19.690068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:19.691711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:19.804345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:19.805783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e20eeb-373d-44cf-b61b-ef4c251c5148 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:19.824117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 727/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:29.457454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=22 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:29.821794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=66 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:39.139639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40db3a-a2b4-4f96-b6c2-37d05ff282ff None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:39.140601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40db3a-a2b4-4f96-b6c2-37d05ff282ff None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:39.140601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40db3a-a2b4-4f96-b6c2-37d05ff282ff None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:39.142516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 744/1471] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:39.147854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506b50-eaea-42cb-95f9-400167bad82e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:39.148330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506b50-eaea-42cb-95f9-400167bad82e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:39.148429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506b50-eaea-42cb-95f9-400167bad82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:39.217198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-19506b50-eaea-42cb-95f9-400167bad82e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:39.218335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-19506b50-eaea-42cb-95f9-400167bad82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:39.319109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 728/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:39.405449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11ce26-0474-44c6-9a41-ce7e1357e733 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:39.405784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11ce26-0474-44c6-9a41-ce7e1357e733 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:39.406129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11ce26-0474-44c6-9a41-ce7e1357e733 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:39.408120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 745/1473] 10.222.0.30 () {54 vars in 966 bytes} [Fri Aug 5 05:07:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:39.413204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a82fc-656e-4521-8831-b852d2a754dd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:39.413488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a82fc-656e-4521-8831-b852d2a754dd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:39.413766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a82fc-656e-4521-8831-b852d2a754dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:39.493464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ed7a82fc-656e-4521-8831-b852d2a754dd None None] MFA Rules not processed for user `a88bb908a0104cc589b19095091a8a53`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:39.494621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ed7a82fc-656e-4521-8831-b852d2a754dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:39.515733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:39.578230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 729/1474] 10.222.0.30 () {58 vars in 1063 bytes} [Fri Aug 5 05:07:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:07:39.586894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11ce26-0474-44c6-9a41-ce7e1357e733 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:39.588460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e11ce26-0474-44c6-9a41-ce7e1357e733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:39.696508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11ce26-0474-44c6-9a41-ce7e1357e733 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:39.773782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:39.774357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:39.774636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:39.774909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:39.776548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:39.834244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:39.893111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:39.894603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62846f3a-24f8-40d0-85e9-150453326cd0 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:39.913552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 746/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:41.964768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a99c04-0bef-42a5-8ab2-7a6e6a240724 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:41.965346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a99c04-0bef-42a5-8ab2-7a6e6a240724 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:41.965346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a99c04-0bef-42a5-8ab2-7a6e6a240724 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:41.966692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 730/1476] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:41.972250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58b8e-575d-4bf2-b0e5-fc23fb06ce87 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:41.972678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58b8e-575d-4bf2-b0e5-fc23fb06ce87 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:41.972791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58b8e-575d-4bf2-b0e5-fc23fb06ce87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:42.042872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5ca58b8e-575d-4bf2-b0e5-fc23fb06ce87 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:42.044425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5ca58b8e-575d-4bf2-b0e5-fc23fb06ce87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:42.157891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 747/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:42.181479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a99c04-0bef-42a5-8ab2-7a6e6a240724 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:42.183024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a1a99c04-0bef-42a5-8ab2-7a6e6a240724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:42.311157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a99c04-0bef-42a5-8ab2-7a6e6a240724 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:42.370242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:42.415687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:42.416331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:42.416607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:42.416877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:42.417563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:42.419309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:07:42.419309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:07:42.419309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:07:42.420472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8341a6-874e-4cf7-905e-1f4f4eccde88 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:42.565217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 731/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Fri Aug 5 05:07:42 2022] GET /identity/v3/projects => generated 2553 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:07:44.562508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec734c6-804f-4fd3-bcab-b8e7511bbba4 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:44.563218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec734c6-804f-4fd3-bcab-b8e7511bbba4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:44.563218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec734c6-804f-4fd3-bcab-b8e7511bbba4 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:44.564780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 748/1479] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:44.570233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909a7e2-115e-4de2-9e5c-0a4a6909069a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:44.570681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909a7e2-115e-4de2-9e5c-0a4a6909069a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:44.570792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909a7e2-115e-4de2-9e5c-0a4a6909069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:44.640322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8909a7e2-115e-4de2-9e5c-0a4a6909069a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:44.641540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8909a7e2-115e-4de2-9e5c-0a4a6909069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:44.747329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 732/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:44.793880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec734c6-804f-4fd3-bcab-b8e7511bbba4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:44.796286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eec734c6-804f-4fd3-bcab-b8e7511bbba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:44.812104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:44.958026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec734c6-804f-4fd3-bcab-b8e7511bbba4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:45.041130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:45.074400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:45.075019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:45.075292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:45.075560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:45.077208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:45.167628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:45.169277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93ee7d-e3c2-427f-9c7b-46f8b633c494 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:45.188024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 749/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:47.330031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15d4a-a186-4890-afd8-87b53d8be6bd None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:47.330601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15d4a-a186-4890-afd8-87b53d8be6bd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:47.330601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a15d4a-a186-4890-afd8-87b53d8be6bd None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:47.331902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 733/1482] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:47.338201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85058de1-255c-4628-b64c-0c1986ff35bf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:47.338931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85058de1-255c-4628-b64c-0c1986ff35bf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:47.339105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85058de1-255c-4628-b64c-0c1986ff35bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:47.417610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-85058de1-255c-4628-b64c-0c1986ff35bf None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:47.418822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-85058de1-255c-4628-b64c-0c1986ff35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:47.466328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:47.520302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 750/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:47.556658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15d4a-a186-4890-afd8-87b53d8be6bd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:47.558180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-72a15d4a-a186-4890-afd8-87b53d8be6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:47.581977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:47.664668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a15d4a-a186-4890-afd8-87b53d8be6bd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:47.742496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:47.743109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:47.743384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:47.743653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:47.745447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:47.829452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:47.867264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:47.868664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbd337-3748-44d9-bd07-95ddb87c3680 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:47.888443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 734/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:49.897864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffa368-0077-4403-80cb-4ab80492d596 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:49.898461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffa368-0077-4403-80cb-4ab80492d596 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:49.898461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffa368-0077-4403-80cb-4ab80492d596 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:49.899797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 751/1485] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:07:49.906122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d945ffd-4e94-4469-8cf3-e074bf6df47f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:49.906579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d945ffd-4e94-4469-8cf3-e074bf6df47f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:49.906736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d945ffd-4e94-4469-8cf3-e074bf6df47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:49.976274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5d945ffd-4e94-4469-8cf3-e074bf6df47f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:49.977493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5d945ffd-4e94-4469-8cf3-e074bf6df47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:50.080964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 735/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:50.133176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffa368-0077-4403-80cb-4ab80492d596 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:50.135431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cdffa368-0077-4403-80cb-4ab80492d596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:50.272578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffa368-0077-4403-80cb-4ab80492d596 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:50.351335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:50.353404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:50.354055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:50.354638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:50.355109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:50.357382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:50.474307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:50.475806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2377-f19c-4d9b-9927-c8f812a2d701 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:50.494236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 752/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:52.579423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7ed73-a58f-4b32-aab8-cb67b3ad4542 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:52.580021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7ed73-a58f-4b32-aab8-cb67b3ad4542 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:52.580021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7ed73-a58f-4b32-aab8-cb67b3ad4542 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:52.581383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 736/1488] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:52.586902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21987014-aae4-475f-8a6d-55200208546f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:52.587351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21987014-aae4-475f-8a6d-55200208546f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:52.587464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21987014-aae4-475f-8a6d-55200208546f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:52.655017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-21987014-aae4-475f-8a6d-55200208546f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:52.656278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-21987014-aae4-475f-8a6d-55200208546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:52.709139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:52.755537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 753/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:52.804037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7ed73-a58f-4b32-aab8-cb67b3ad4542 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:52.805672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-46c7ed73-a58f-4b32-aab8-cb67b3ad4542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:52.843706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:52.916506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7ed73-a58f-4b32-aab8-cb67b3ad4542 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:52.996497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:52.997095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:52.997403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:52.997711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:52.999337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:53.086126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:53.118640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:53.120097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15c685-58a6-4c3b-95de-423917ad15b1 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:53.138261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 737/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:07:55.317672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d28a-e53a-459f-9c75-37275a230520 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:55.318213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d28a-e53a-459f-9c75-37275a230520 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:55.318213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d28a-e53a-459f-9c75-37275a230520 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:55.319505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 754/1491] 10.222.0.30 () {54 vars in 927 bytes} [Fri Aug 5 05:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:07:55.325049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7829a66-2d27-4c5a-bde7-ebd02b0c3138 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:55.325519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7829a66-2d27-4c5a-bde7-ebd02b0c3138 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:55.325669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7829a66-2d27-4c5a-bde7-ebd02b0c3138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:55.395473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f7829a66-2d27-4c5a-bde7-ebd02b0c3138 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:07:55.396715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f7829a66-2d27-4c5a-bde7-ebd02b0c3138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:55.499482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 738/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Fri Aug 5 05:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:07:55.539208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d28a-e53a-459f-9c75-37275a230520 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:07:55.540772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ba09d28a-e53a-459f-9c75-37275a230520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:55.652671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d28a-e53a-459f-9c75-37275a230520 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:07:55.729617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:07:55.730262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:07:55.730544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:07:55.730817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:07:55.732471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:07:55.743176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:07:55.850249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:07:55.851709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a81474-8045-4e7a-8004-394ef15d7c63 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:07:55.870283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 755/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:08:05.497609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=82 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:08:05.868084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=50 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:35.550337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:35.550971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:35.550971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:35.630641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:09:35.631892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:35.729889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 739/1494] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:09:35.741746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:35.742305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:35.742305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:35.819710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:09:35.820955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:35.921764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 756/1495] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:09:35.927470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:35.929316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:35.947393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b786-51a7-4156-bd17-15dd5dc674b1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:36.015453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:36.050583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:36.051205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.051483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.051751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:36.052471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:09:36.052775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:36.053801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:36.053801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:09:36.053801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:09:36.054687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:36.066546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 740/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:09:36.075485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:36.085241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625017f-8084-419e-8b15-bc5abb64580b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:36.152203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:36.188269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:36.188897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.189175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.189446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:36.190112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:36.191880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:36.191880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:09:36.191880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:09:36.193088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:36.252750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 757/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:09:36 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:09:36.257180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:36.258988 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:36.293458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71f28a-aa23-4500-822e-be0bfc144487 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:36.395053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:36.395685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.395946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.396251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:36.396911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:36.397926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:36.397926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:09:36.397926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:09:36.399123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:36.447265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 741/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:09:36 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:09:36.452069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:36.453659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:36.520583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa88a8-6dcb-4680-94b8-0b31cb994e79 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:36.613510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:36.625525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:36.626108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.626388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.626653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:36.627382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:36.628567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:36.628567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:09:36.628567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:09:36.629450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:36.642541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 758/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:09:36.646768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:36.648326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:36.659987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93afb07-d1fa-4c32-9671-b6d2628b069d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:36.686153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:36.760878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:36.761511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.761796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.762034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:36.762740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:36.763799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:36.763799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:09:36.763799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:09:36.764712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf50e8-50c0-4169-b9e9-c9b6adb02da7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:36.778603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 742/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:09:36.783307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:36.784933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:36.821436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d83d2-5c2e-45db-8c12-cdb1ef0919e9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:36.916228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:36.916898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.917170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.917441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] PATH_INFO: `/v3/projects/ae0ea59f837e4d699f011507bda67aae/users/4f9b3f192b9f42b3ba29549981287c48/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:36.940482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0ea59f837e4d699f011507bda67aae, user_id=4f9b3f192b9f42b3ba29549981287c48, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:36.941513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:36.941513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:09:36.941513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:09:36.943434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646b411-2610-4454-b12c-f89240615a7f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:36.971582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 759/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:09:36 2022] PUT /identity/v3/projects/ae0ea59f837e4d699f011507bda67aae/users/4f9b3f192b9f42b3ba29549981287c48/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:09:36.984782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfad1-2fd6-4c4f-949a-5b97e933f5cf None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:36.985091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfad1-2fd6-4c4f-949a-5b97e933f5cf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:36.985362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfad1-2fd6-4c4f-949a-5b97e933f5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:37.043956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-386cfad1-2fd6-4c4f-949a-5b97e933f5cf None None] MFA Rules not processed for user `4f9b3f192b9f42b3ba29549981287c48`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:09:37.045540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-386cfad1-2fd6-4c4f-949a-5b97e933f5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:37.124208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 743/1502] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:09:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:09:37.137775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8e703-a5cd-4958-9f94-903bfd2c8ca8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:37.138068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8e703-a5cd-4958-9f94-903bfd2c8ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:37.138343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8e703-a5cd-4958-9f94-903bfd2c8ca8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:37.139686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 760/1503] 10.222.0.30 () {54 vars in 964 bytes} [Fri Aug 5 05:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:09:37.144514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662adb9-ab8d-49b0-bdda-63fd244e2c3e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:37.144832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662adb9-ab8d-49b0-bdda-63fd244e2c3e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:37.145110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662adb9-ab8d-49b0-bdda-63fd244e2c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:37.175936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:37.222349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c662adb9-ab8d-49b0-bdda-63fd244e2c3e None None] MFA Rules not processed for user `ea0805461cd3492fa5e2ebbea407e351`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:09:37.223569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c662adb9-ab8d-49b0-bdda-63fd244e2c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:37.303339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 744/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Fri Aug 5 05:09:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:09:37.314968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8e703-a5cd-4958-9f94-903bfd2c8ca8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:37.316519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4ff8e703-a5cd-4958-9f94-903bfd2c8ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:37.367327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:37.424933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8e703-a5cd-4958-9f94-903bfd2c8ca8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:37.505072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:37.505615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:37.505893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:37.506156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:37.507788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:37.612427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:37.613872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:37.617381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:37.634008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 761/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:09:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:09:37.680531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857517a-0635-4e51-847b-6da1c1d60a17 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:37.680992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857517a-0635-4e51-847b-6da1c1d60a17 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:37.681107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857517a-0635-4e51-847b-6da1c1d60a17 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:37.682567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 745/1506] 10.222.0.30 () {52 vars in 830 bytes} [Fri Aug 5 05:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:09:37.686616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:37.687878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:37.792131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d7ede-6dfb-42e1-a085-1cba2a77d8d3 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:37.888596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:37.889231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:37.889513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:37.889779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] PATH_INFO: `/v3/users/4f9b3f192b9f42b3ba29549981287c48` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:37.902263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9b3f192b9f42b3ba29549981287c48)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:37.903292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:37.903292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:09:37.903292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:09:37.904538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:37.914268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:37.977725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9b3f192b9f42b3ba29549981287c48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:37.978152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9b3f192b9f42b3ba29549981287c48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:37.992329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9b3f192b9f42b3ba29549981287c48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:37.992862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9b3f192b9f42b3ba29549981287c48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:38.004430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9b3f192b9f42b3ba29549981287c48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:38.033691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 762/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:09:37 2022] DELETE /identity/v3/users/4f9b3f192b9f42b3ba29549981287c48 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:09:38.038386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:38.038671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:38.038944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:38.098047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:09:38.099188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:38.184889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:38.198330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 746/1508] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:09:38.209682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:38.210937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:38.311570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74b229-7668-4983-9153-1e5e4b5e6f35 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:38.369224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:38.390111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:38.390736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:38.391018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:38.391290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:38.392931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:38.505685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:38.507143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebd14f-af98-4182-b947-404efaf3c7a0 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:38.525372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 763/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:09:38.916724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:09:38.918527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:09:38.931576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b1e15-11b3-4530-aa91-b51283d7d0db None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:09:39.029316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:09:39.029983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:09:39.030247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:09:39.030520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] PATH_INFO: `/v3/projects/ae0ea59f837e4d699f011507bda67aae` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:09:39.040291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0ea59f837e4d699f011507bda67aae)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:09:39.041404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:09:39.041404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:09:39.041404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:09:39.042594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:09:39.108805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0ea59f837e4d699f011507bda67aae', 'request_id': 'req-3647070e-622a-4785-831e-c359d0fdaf61'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:39.115377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0ea59f837e4d699f011507bda67aae', 'request_id': 'req-3647070e-622a-4785-831e-c359d0fdaf61'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:39.129391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0ea59f837e4d699f011507bda67aae', 'request_id': 'req-3647070e-622a-4785-831e-c359d0fdaf61'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:39.162725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] The token cache is being invalidate because project ae0ea59f837e4d699f011507bda67aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:09:39.163010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:39.163453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3647070e-622a-4785-831e-c359d0fdaf61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:09:39.164949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 747/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:09:38 2022] DELETE /identity/v3/projects/ae0ea59f837e4d699f011507bda67aae => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:09:48.523123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=62 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:09:49.160174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:11:09.310025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a19b61-1b1b-4b73-882f-649c449dc6d9 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:09.311013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a19b61-1b1b-4b73-882f-649c449dc6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:09.311262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a19b61-1b1b-4b73-882f-649c449dc6d9 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:09.312851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:09.318098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49175c48-b77c-4937-ab94-efdf2989dc68 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:09.318524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49175c48-b77c-4937-ab94-efdf2989dc68 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:09.318699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49175c48-b77c-4937-ab94-efdf2989dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:09.407977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-49175c48-b77c-4937-ab94-efdf2989dc68 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:09.409225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-49175c48-b77c-4937-ab94-efdf2989dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:09.526930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:09.678315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a19b61-1b1b-4b73-882f-649c449dc6d9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:09.682226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-73a19b61-1b1b-4b73-882f-649c449dc6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:09.815889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a19b61-1b1b-4b73-882f-649c449dc6d9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:09.822667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0045d-881e-4e6f-afae-8b546d0d0b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:09.823298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0045d-881e-4e6f-afae-8b546d0d0b85 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:09.823583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0045d-881e-4e6f-afae-8b546d0d0b85 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:09.823877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0045d-881e-4e6f-afae-8b546d0d0b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:09.832205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0045d-881e-4e6f-afae-8b546d0d0b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:09.833817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0045d-881e-4e6f-afae-8b546d0d0b85 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:09.843480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 765/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:10.665970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6bb3-f93c-44a0-bae4-9ffc8948746b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:10.666547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff71b5-d9b0-41da-9998-1a6c72787399 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:10.666915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6bb3-f93c-44a0-bae4-9ffc8948746b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:10.667013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff71b5-d9b0-41da-9998-1a6c72787399 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:10.667107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6bb3-f93c-44a0-bae4-9ffc8948746b None None] PATH_INFO: `/v3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:10.667193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff71b5-d9b0-41da-9998-1a6c72787399 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:10.668404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 749/1514] 10.222.0.36 () {54 vars in 925 bytes} [Fri Aug 5 05:11:10 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:11:10.668747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 766/1515] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:10.673044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:10.673331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:10.673610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:10.674835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9013a-5d3d-4a05-85b2-3f22355167e5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:10.675128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9013a-5d3d-4a05-85b2-3f22355167e5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:10.675392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9013a-5d3d-4a05-85b2-3f22355167e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:10.693060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:10.694295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:10.703611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 750/1516] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:10.737072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-49a9013a-5d3d-4a05-85b2-3f22355167e5 None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:10.738278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49a9013a-5d3d-4a05-85b2-3f22355167e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:10.805863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:11:10.811959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 767/1517] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Aug 5 05:11:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:10.854943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:10.867539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb8b95-fec0-4f7e-9926-153a85649371 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:10.874013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e3cd1-0e35-47ce-9e48-d4466869b8e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:10.874630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e3cd1-0e35-47ce-9e48-d4466869b8e4 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:10.874914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e3cd1-0e35-47ce-9e48-d4466869b8e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:10.875191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e3cd1-0e35-47ce-9e48-d4466869b8e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:10.883413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e3cd1-0e35-47ce-9e48-d4466869b8e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:10.885072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e3cd1-0e35-47ce-9e48-d4466869b8e4 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:10.894595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 751/1518] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:19.646584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3460d-ab19-44f4-8be6-f549b6693dfb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:19.647119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3460d-ab19-44f4-8be6-f549b6693dfb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:19.647119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3460d-ab19-44f4-8be6-f549b6693dfb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:19.648441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 768/1519] 10.222.0.30 () {54 vars in 961 bytes} [Fri Aug 5 05:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:19.654498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5da3-0e94-4dda-871c-bbfc663abcc1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:19.655240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5da3-0e94-4dda-871c-bbfc663abcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:19.655240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c5da3-0e94-4dda-871c-bbfc663abcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:19.685732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-528c5da3-0e94-4dda-871c-bbfc663abcc1 None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:19.686969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-528c5da3-0e94-4dda-871c-bbfc663abcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:19.697145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 752/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Fri Aug 5 05:11:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:19.704927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3460d-ab19-44f4-8be6-f549b6693dfb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:19.720377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3460d-ab19-44f4-8be6-f549b6693dfb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:19.726425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f032a1-1304-43c6-a26e-982cf34eb57e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:19.727015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f032a1-1304-43c6-a26e-982cf34eb57e service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:19.727299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f032a1-1304-43c6-a26e-982cf34eb57e service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:19.727569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f032a1-1304-43c6-a26e-982cf34eb57e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:19.735970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f032a1-1304-43c6-a26e-982cf34eb57e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:19.737545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f032a1-1304-43c6-a26e-982cf34eb57e service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:19.746858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 769/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:22.222522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549a03-d248-44ee-ad44-77536b9a344e None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:22.223079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549a03-d248-44ee-ad44-77536b9a344e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:22.223175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549a03-d248-44ee-ad44-77536b9a344e None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:22.225269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:11:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:22.232551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5dd34-766a-4a0e-a581-de25663ee986 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:22.233013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5dd34-766a-4a0e-a581-de25663ee986 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:22.233105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5dd34-766a-4a0e-a581-de25663ee986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:22.254271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-99b5dd34-766a-4a0e-a581-de25663ee986 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:22.255391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-99b5dd34-766a-4a0e-a581-de25663ee986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:22.264194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:11:22.419362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b549a03-d248-44ee-ad44-77536b9a344e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:22.437191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b549a03-d248-44ee-ad44-77536b9a344e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:22.443654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de36f8a-d8a7-4289-8433-6431df45a0e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:22.444296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de36f8a-d8a7-4289-8433-6431df45a0e2 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:22.444576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de36f8a-d8a7-4289-8433-6431df45a0e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:22.444848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de36f8a-d8a7-4289-8433-6431df45a0e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:22.453377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de36f8a-d8a7-4289-8433-6431df45a0e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:22.454959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de36f8a-d8a7-4289-8433-6431df45a0e2 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:22.464443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 754/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:23.273358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c84cf-f502-4ecb-95dc-52e7ba56490e None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:23.273940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c84cf-f502-4ecb-95dc-52e7ba56490e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:23.273940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c84cf-f502-4ecb-95dc-52e7ba56490e None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:23.275299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 05:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:23.281148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebeca7-6395-48b3-93d3-53f013c86b7a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:23.281953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebeca7-6395-48b3-93d3-53f013c86b7a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:23.281953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebeca7-6395-48b3-93d3-53f013c86b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:23.306732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2eebeca7-6395-48b3-93d3-53f013c86b7a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:23.307992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2eebeca7-6395-48b3-93d3-53f013c86b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:23.317187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 05:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:23.468317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c84cf-f502-4ecb-95dc-52e7ba56490e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:23.485263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c84cf-f502-4ecb-95dc-52e7ba56490e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:23.492162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ede999-8322-47a6-9efe-5d2102d202f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:23.492765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ede999-8322-47a6-9efe-5d2102d202f8 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:23.493047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ede999-8322-47a6-9efe-5d2102d202f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:23.493323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ede999-8322-47a6-9efe-5d2102d202f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:23.502083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ede999-8322-47a6-9efe-5d2102d202f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:23.503674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ede999-8322-47a6-9efe-5d2102d202f8 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:23.513131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 772/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:27.349379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff037e70-b7be-4232-9a97-3d0a340dfe38 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:27.349934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff037e70-b7be-4232-9a97-3d0a340dfe38 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:27.349934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff037e70-b7be-4232-9a97-3d0a340dfe38 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:27.351308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:27.357487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7adf-6657-430c-be39-05146e379d37 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:27.357937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7adf-6657-430c-be39-05146e379d37 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:27.358044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf7adf-6657-430c-be39-05146e379d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:27.382946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bdaf7adf-6657-430c-be39-05146e379d37 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:27.384199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bdaf7adf-6657-430c-be39-05146e379d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:27.392876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:27.469270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff037e70-b7be-4232-9a97-3d0a340dfe38 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:27.471224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ff037e70-b7be-4232-9a97-3d0a340dfe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:27.498422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:11:27.551285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff037e70-b7be-4232-9a97-3d0a340dfe38 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:27.555354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf37740-ad1d-41be-ab5b-01a22ee62f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:27.556006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf37740-ad1d-41be-ab5b-01a22ee62f5b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:27.556279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf37740-ad1d-41be-ab5b-01a22ee62f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:27.556552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf37740-ad1d-41be-ab5b-01a22ee62f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:27.566601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf37740-ad1d-41be-ab5b-01a22ee62f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:27.568059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf37740-ad1d-41be-ab5b-01a22ee62f5b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:27.577715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 757/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:29.569864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a312-da5a-4919-9411-c8853d712786 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:29.570458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a312-da5a-4919-9411-c8853d712786 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:29.570458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1a312-da5a-4919-9411-c8853d712786 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:29.571844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:29.578302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a32600-7164-4752-b8de-5e19ce753378 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:29.579049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a32600-7164-4752-b8de-5e19ce753378 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:29.579201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a32600-7164-4752-b8de-5e19ce753378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:29.603249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-21a32600-7164-4752-b8de-5e19ce753378 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:29.604470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-21a32600-7164-4752-b8de-5e19ce753378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:29.613439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:29.649013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1a312-da5a-4919-9411-c8853d712786 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:29.661899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1a312-da5a-4919-9411-c8853d712786 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:29.666109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63f486-1837-472a-bc70-f4dc652ebb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:29.666718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63f486-1837-472a-bc70-f4dc652ebb65 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:29.667003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63f486-1837-472a-bc70-f4dc652ebb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:29.667279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63f486-1837-472a-bc70-f4dc652ebb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:29.677197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63f486-1837-472a-bc70-f4dc652ebb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:29.678638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63f486-1837-472a-bc70-f4dc652ebb65 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:29.688190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 775/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:33.185711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f049-ea8a-470f-8fa3-56c0fd804086 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:33.186628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f049-ea8a-470f-8fa3-56c0fd804086 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:33.186628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f049-ea8a-470f-8fa3-56c0fd804086 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:33.188942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:11:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:33.197771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e1ec3-7b39-4905-a21f-92fca716e7b4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:33.198505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e1ec3-7b39-4905-a21f-92fca716e7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:33.198642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e1ec3-7b39-4905-a21f-92fca716e7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:33.224717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-415e1ec3-7b39-4905-a21f-92fca716e7b4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:33.225925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-415e1ec3-7b39-4905-a21f-92fca716e7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:33.234979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:33.536186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f049-ea8a-470f-8fa3-56c0fd804086 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:33.539397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9231f049-ea8a-470f-8fa3-56c0fd804086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:33.631468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f049-ea8a-470f-8fa3-56c0fd804086 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:33.638133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f9eda-dd6c-44d5-b00b-51d2249ff156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:33.638760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9eda-dd6c-44d5-b00b-51d2249ff156 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:33.639038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9eda-dd6c-44d5-b00b-51d2249ff156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:33.639306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9eda-dd6c-44d5-b00b-51d2249ff156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:33.649302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9eda-dd6c-44d5-b00b-51d2249ff156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:33.650730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9eda-dd6c-44d5-b00b-51d2249ff156 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:33.660256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 760/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:35.685720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0919a-6ad2-4133-b8da-d87ed776f4aa None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:35.686333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0919a-6ad2-4133-b8da-d87ed776f4aa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:35.686333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0919a-6ad2-4133-b8da-d87ed776f4aa None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:35.687634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 05:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:11:35.693446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d328239-5579-4dd9-b153-653d74c45d77 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:35.693871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d328239-5579-4dd9-b153-653d74c45d77 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:35.694001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d328239-5579-4dd9-b153-653d74c45d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:35.716260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0d328239-5579-4dd9-b153-653d74c45d77 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:11:35.717471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0d328239-5579-4dd9-b153-653d74c45d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:35.728384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 05:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:11:36.034265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0919a-6ad2-4133-b8da-d87ed776f4aa None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:11:36.037327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d2c0919a-6ad2-4133-b8da-d87ed776f4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:11:36.059496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0919a-6ad2-4133-b8da-d87ed776f4aa None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:11:36.063703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd40970-15d7-483e-a928-d5ecf1aef891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:11:36.064346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd40970-15d7-483e-a928-d5ecf1aef891 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:11:36.064631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd40970-15d7-483e-a928-d5ecf1aef891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:11:36.064908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd40970-15d7-483e-a928-d5ecf1aef891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:11:36.074091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd40970-15d7-483e-a928-d5ecf1aef891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:11:36.075532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd40970-15d7-483e-a928-d5ecf1aef891 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:11:36.085193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 778/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:11:45.710114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=66 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:11:46.081283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=34 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:58.075272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7a1f-8c1f-493d-8464-7a4af0ffc6ed None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.075960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7a1f-8c1f-493d-8464-7a4af0ffc6ed None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.075960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7a1f-8c1f-493d-8464-7a4af0ffc6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.101172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-834a7a1f-8c1f-493d-8464-7a4af0ffc6ed None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.102779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-834a7a1f-8c1f-493d-8464-7a4af0ffc6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.114732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 762/1540] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.124535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.125158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.125158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.148789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26b6bc-9490-40c1-96e8-bc5f99ad255d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.149467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26b6bc-9490-40c1-96e8-bc5f99ad255d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.150004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26b6bc-9490-40c1-96e8-bc5f99ad255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.156879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.158578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.178088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 779/1541] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.186669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.187158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-cc26b6bc-9490-40c1-96e8-bc5f99ad255d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.188682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-cc26b6bc-9490-40c1-96e8-bc5f99ad255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.199447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2facdbb-93f7-459c-b097-cfc8574ff31c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.205626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 763/1542] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:12:58.212972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.213655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.213939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.214486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.215236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:58.215549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.216654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:12:58.216654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:12:58.216654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:12:58.217392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.217550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e40e-9859-4ab8-b7f4-2cf4203fca27 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.218780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.219299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.237509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 780/1543] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:58.251585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.251805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.252101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.261105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.262760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.286445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 764/1544] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.287664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.288133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.289381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.308697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 781/1545] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:12:58.309581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.310905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b693b8b-eb4e-4eae-bda8-d0abd31b132a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.324497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.325568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.325978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50dcb78-dce5-4535-8f14-3349fc74fb0b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.326299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.327055 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.327956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.329216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:12:58.329216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:12:58.329216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:12:58.330904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.341588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.342466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.342751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.343079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.344042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:58.344369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.346421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cda149-1a88-457f-ac78-262b31738b9e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.372854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 782/1546] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:58.374198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.374494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.374776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.409011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.410308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.425013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 783/1547] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.426015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.437301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de0c2f-1fec-4315-97f8-a999e196cbc2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.437894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 765/1548] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:58.438996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.536896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.537972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.538321 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.538659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.539366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.543818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.582805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 784/1549] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:58.583705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.594188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af6e00-e07f-45e8-bd78-7ede16ac96bc admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.662225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.662872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.663154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.663426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.664156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:58.664462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.666472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cd2ae-8518-42ee-ae87-b5559a97d4ab admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.668939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a40b3-7d5c-4689-bccc-17e24eac6c9e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.674032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.674662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.675399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.675689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.676400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.678081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 766/1550] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:58.678440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:12:58.678440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:12:58.678440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:12:58.679433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53b87-473a-4349-b812-e8629fe036d2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.679665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44815365-d6ad-4848-83ad-f8426058db94 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.679798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53b87-473a-4349-b812-e8629fe036d2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.680022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53b87-473a-4349-b812-e8629fe036d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.701420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-18f53b87-473a-4349-b812-e8629fe036d2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.702647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-18f53b87-473a-4349-b812-e8629fe036d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.704437 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 785/1551] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:58.705648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.705942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.706222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.711162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 767/1552] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.712425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96814a55-fd6b-4f65-951d-3824c80f8786 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.712709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96814a55-fd6b-4f65-951d-3824c80f8786 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.712983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96814a55-fd6b-4f65-951d-3824c80f8786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.732267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.733569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.734788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-96814a55-fd6b-4f65-951d-3824c80f8786 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.736066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-96814a55-fd6b-4f65-951d-3824c80f8786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.741985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 786/1553] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.742948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.744689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 768/1554] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.745756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de70ef-ae93-436d-a6ee-57281bb20ec8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.746028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de70ef-ae93-436d-a6ee-57281bb20ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.746302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de70ef-ae93-436d-a6ee-57281bb20ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.753333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d682c-62bd-41c3-87c9-a8b2df1b29af None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.760070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.760700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.760980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.761253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.761924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.763893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.766351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-11de70ef-ae93-436d-a6ee-57281bb20ec8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.767541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-11de70ef-ae93-436d-a6ee-57281bb20ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.775800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 769/1555] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.779654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.779887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.780221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.787596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 787/1556] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/users => generated 286 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:58.788379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.794953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:58.799061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e1e77-c444-416c-afbd-29a7e16acca6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.800994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.802404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.805476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7e9b1-fe37-4318-ad0e-c0e407252b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.806088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7e9b1-fe37-4318-ad0e-c0e407252b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.806367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7e9b1-fe37-4318-ad0e-c0e407252b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.806664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7e9b1-fe37-4318-ad0e-c0e407252b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.807371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7e9b1-fe37-4318-ad0e-c0e407252b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.809462 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7e9b1-fe37-4318-ad0e-c0e407252b99 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.811409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 770/1557] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.812350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.823407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4560a2e-2930-4a61-8e89-429d92f0511a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.829537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253db30c-25f9-4e42-a260-3a4ad7580288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.830505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253db30c-25f9-4e42-a260-3a4ad7580288 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.830878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253db30c-25f9-4e42-a260-3a4ad7580288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.831300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253db30c-25f9-4e42-a260-3a4ad7580288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.832105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253db30c-25f9-4e42-a260-3a4ad7580288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.833844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253db30c-25f9-4e42-a260-3a4ad7580288 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.837479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 788/1558] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:58.838598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e81dc3-bbff-4ff2-ba91-431a382eeb5d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.838878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e81dc3-bbff-4ff2-ba91-431a382eeb5d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.839156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e81dc3-bbff-4ff2-ba91-431a382eeb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.845748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 771/1559] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:58.847431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651dca-cdf1-439e-a304-c5355acbbe3a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.847717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651dca-cdf1-439e-a304-c5355acbbe3a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.848018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651dca-cdf1-439e-a304-c5355acbbe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.860587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71e81dc3-bbff-4ff2-ba91-431a382eeb5d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.862008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71e81dc3-bbff-4ff2-ba91-431a382eeb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.866462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4a651dca-cdf1-439e-a304-c5355acbbe3a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.867643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4a651dca-cdf1-439e-a304-c5355acbbe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.875919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:58.920257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 789/1560] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.921519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.921808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.922081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.932274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 772/1561] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.933593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98e918-239c-4c07-b478-68c61027386c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.933880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98e918-239c-4c07-b478-68c61027386c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.934204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98e918-239c-4c07-b478-68c61027386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.942619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.943972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.952264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 790/1562] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.953035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.953174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7f98e918-239c-4c07-b478-68c61027386c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.954286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7f98e918-239c-4c07-b478-68c61027386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.962322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 773/1563] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:58.962744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dc3a4-9947-44fd-ac3d-03af971046f5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.963510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.963829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.964123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.967859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.968469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.968752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.969026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.969720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:58.971320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:58.981488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 791/1564] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:58.982261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:58.985516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:58.986707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:58.991807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d29544-246c-4b2f-8cd0-50d03eb1026d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:58.996304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:58.996908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:58.997191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:58.997464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:58.998121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.002404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 774/1565] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:59.002404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.003259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.009590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6dec1-40d1-4f80-b37e-7007dacb52fe None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.015264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.015900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.016231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.016484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.017133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.018855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.024632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 792/1566] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:58 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.025659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.035933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 775/1567] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.035933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.036335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419718ee-d2a9-4822-86e2-f599a9f484ab admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.039955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.040588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.040842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.041464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.042178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:59.042486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.044176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.046453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d19a3-5e0c-4e94-b561-a8ad4447a301 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.051125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.051855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.052131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.052398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.053090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:59.053392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.054977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.056454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 793/1568] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:59.057269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.066822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e2614-9304-4d95-9de8-0b5602cb271c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.072402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.073004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.073288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.073563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.074231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:59.074367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 776/1569] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:59.074540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.075096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.076166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.086325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fca0e-e207-4d16-b6ff-ee4c2bce1820 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.088643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 794/1570] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:59.089380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.097460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.098107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.098381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.098697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.099381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:59.099682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.101015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa3c6-64c6-4db9-96b9-8e68a3b8ae1a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.101308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.105955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.106564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.106840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.107116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.107856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.109427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.113313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 777/1571] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:59.114212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.120069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 795/1572] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.120889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.127911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a82f8-32a4-4f5d-9ae0-7ba4bd8d0813 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.130342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a86c95-a3e4-45d8-b930-22ea5c30f1f4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.132463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.133076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.133360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.133636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.134319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:12:59.134625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.135081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.135804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.136311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.136426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.136527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.137204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.138837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.149200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 778/1573] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:12:59.149528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 796/1574] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.150308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.150475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.165058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100a74e-d871-485b-bc88-00d3a7f28ef1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.165759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515d57-3633-49da-b8fd-d0ec66d0dc96 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.169930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.170593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.170879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.171158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] PATH_INFO: `/v3/projects/08e98cefc4d147bba944a1cf82f2e772/users/28432f2e1dea486cb163fa2dab6e443f/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.171694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.172340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.172619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.172891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.173554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.175472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.186359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e98cefc4d147bba944a1cf82f2e772, user_id=28432f2e1dea486cb163fa2dab6e443f, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.187403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:12:59.187403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:12:59.187403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:12:59.189782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.205534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 779/1575] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:12:59 2022] PUT /identity/v3/projects/08e98cefc4d147bba944a1cf82f2e772/users/28432f2e1dea486cb163fa2dab6e443f/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:12:59.205695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 797/1576] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.206332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.210054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.250958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:59.268976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4318-cc61-4d33-a521-5160c90320a8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.272844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aca4ef-b24b-4703-a624-42b3d42b2780 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.273743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.274440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.274787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.275062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.275722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.277734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.278104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.278721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.278998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.279268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.279959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.281855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.308181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 780/1577] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.308181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.313887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 798/1578] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.313887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.323282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:59.373130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305c66-d24c-4b39-bf47-57b98c42cc87 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.373397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17102a-c1b3-42f5-97e7-fb7f3e8ad2f0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.377759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.378002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.378593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.378708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.378941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.379212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.379317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] PATH_INFO: `/v3/projects/d55cdc16c75f4fda92582251296ae60c/users/0c27df6b5e8042e2be0092d727a2344f/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.379673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.380455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.382420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.394649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55cdc16c75f4fda92582251296ae60c, user_id=0c27df6b5e8042e2be0092d727a2344f, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.397414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.411618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 799/1579] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:12:59 2022] PUT /identity/v3/projects/d55cdc16c75f4fda92582251296ae60c/users/0c27df6b5e8042e2be0092d727a2344f/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:12:59.423381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 781/1580] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.423532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.423630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.473214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fae5f-e3d7-4b1e-8878-75b2bc0c8ef2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.477886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.478514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.478674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2179ed0-a307-4b2e-985e-886d6e95ddf5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.478785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.479061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.479750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.481385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.483493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51410a0-975c-42ff-ae27-90040eda6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.483901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51410a0-975c-42ff-ae27-90040eda6915 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.484184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51410a0-975c-42ff-ae27-90040eda6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.484458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51410a0-975c-42ff-ae27-90040eda6915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.485242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51410a0-975c-42ff-ae27-90040eda6915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.487160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51410a0-975c-42ff-ae27-90040eda6915 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.491340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 782/1581] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.492023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.501187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ea631-b3c9-48c8-9d68-2fa4ac0c4134 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.506374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.507171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.507471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.507913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.508634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.511000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.518380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 800/1582] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.520277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.520952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.521514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.536807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 783/1583] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.536807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.540865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:59.542583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:59.546634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728a22c-48ec-40c4-a254-50defa099c84 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.567969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:59.600639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 801/1584] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:59.601451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.606535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.607431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.607574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.607864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.608529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.610614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.611328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9617ba3-bea4-4b69-87c4-87e16ce6dfd0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.615912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b542ca-ba00-46ce-a4a9-7a5978fa9b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.616523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b542ca-ba00-46ce-a4a9-7a5978fa9b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.616812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b542ca-ba00-46ce-a4a9-7a5978fa9b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.617109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b542ca-ba00-46ce-a4a9-7a5978fa9b5f admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.617822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b542ca-ba00-46ce-a4a9-7a5978fa9b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.619991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b542ca-ba00-46ce-a4a9-7a5978fa9b5f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.635551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 784/1585] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.635551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.639826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:59.645814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 802/1586] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.645814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.646081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19e8f6-87af-4242-86eb-6104fc42f132 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.646198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.646198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.652043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.652750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.653210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.653544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.654221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.656189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.672304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] MFA Rules not processed for user `0c27df6b5e8042e2be0092d727a2344f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:59.673533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:59.681986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 785/1587] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/users => generated 300 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.682602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.692238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237888b-884d-47b8-9d3c-5b101a6c8576 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.696910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.697605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.697890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.698264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] PATH_INFO: `/v3/projects/bdb0ab67f8e54568a8be4a7d80411863/users/16d408692a9f4b689c0a12e03e0f8e05/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.714239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb0ab67f8e54568a8be4a7d80411863, user_id=16d408692a9f4b689c0a12e03e0f8e05, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.716952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.731351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 786/1588] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:12:59 2022] PUT /identity/v3/projects/bdb0ab67f8e54568a8be4a7d80411863/users/16d408692a9f4b689c0a12e03e0f8e05/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:12:59.732169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.738689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 803/1589] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:12:59.739789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.753487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28afe62-0051-423b-9cd1-fff53047fb97 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.809480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705d36d-7601-42e7-b886-8dc47d527767 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.812431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.813086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.813330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.813606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.814304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.814419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.815147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.815457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.815730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.815958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.816433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.819152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.825810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 804/1590] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.826670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.841910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 787/1591] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/users => generated 279 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:12:59.842854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.852373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5f12-b08d-4851-9ee6-45a18fa0f2ee admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.859273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.859914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.860190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.860457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.861150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.862734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.869858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:12:59.873870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 788/1592] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.874702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.885139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731ac1-1491-496a-9e5d-2912ce71ce38 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.887873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb80c9-47e4-4321-b659-3118af5e7f3c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.889793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.890401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.890684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.890956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.891644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.892016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785e495-dae5-4f42-8e44-978295ed9652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.892711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e495-dae5-4f42-8e44-978295ed9652 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.892997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e495-dae5-4f42-8e44-978295ed9652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.893264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.893366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e495-dae5-4f42-8e44-978295ed9652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.902785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785e495-dae5-4f42-8e44-978295ed9652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.903734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 789/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.904272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785e495-dae5-4f42-8e44-978295ed9652 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.904542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.914304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 805/1594] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:12:59.915179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f41425d-761a-437e-b2f6-edc026036e67 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.915515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.915837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.916526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.920369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a61b5d-8d09-47bd-8504-a0960ab7fdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.920974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a61b5d-8d09-47bd-8504-a0960ab7fdee admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.921250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a61b5d-8d09-47bd-8504-a0960ab7fdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.921521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a61b5d-8d09-47bd-8504-a0960ab7fdee admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.922208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a61b5d-8d09-47bd-8504-a0960ab7fdee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.923824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a61b5d-8d09-47bd-8504-a0960ab7fdee admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.934544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 790/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.935650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3673e8b-79cc-41f1-b976-251e0a7133b9 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.935968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3673e8b-79cc-41f1-b976-251e0a7133b9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.936239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3673e8b-79cc-41f1-b976-251e0a7133b9 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.941783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 791/1596] 10.222.0.30 () {54 vars in 962 bytes} [Fri Aug 5 05:12:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:12:59.942488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3673e8b-79cc-41f1-b976-251e0a7133b9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.945971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] MFA Rules not processed for user `16d408692a9f4b689c0a12e03e0f8e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:12:59.947204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:12:59.957566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3673e8b-79cc-41f1-b976-251e0a7133b9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.962373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.962997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.963275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.963545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.964287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.965900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:12:59.979627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 792/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:12:59.980549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:12:59.990450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40789d78-778f-46fe-833e-fae5a7f4603b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:12:59.995016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:12:59.995639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:12:59.996259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:12:59.996259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:12:59.996907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:12:59.998519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.009620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 793/1598] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:00.014468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.023247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687daa56-628a-4c42-b57f-1e36894a7f0e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.029769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.031179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.031563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.031997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.032839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.034911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.040312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 806/1599] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:00.041558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.048623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 794/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:00.049488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.052632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdcc54-6158-4d38-83ed-88bd212dcb03 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.057182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b89e4-cbdb-4f16-8ecf-dd0ccf18332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.057842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b89e4-cbdb-4f16-8ecf-dd0ccf18332d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.058090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b89e4-cbdb-4f16-8ecf-dd0ccf18332d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.058366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b89e4-cbdb-4f16-8ecf-dd0ccf18332d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.059063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b89e4-cbdb-4f16-8ecf-dd0ccf18332d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.060713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b89e4-cbdb-4f16-8ecf-dd0ccf18332d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.064813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2d103-4a72-47ac-bb48-c9f28725f6f0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.070730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.071075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.071197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 807/1601] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:00.071361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.071635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.072455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.072614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.072736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.073016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.074397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.095610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 795/1602] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:00.100707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.111923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b044e-1739-47fe-be9b-188e51fa99d1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.118109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.118939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.119291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.119628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] PATH_INFO: `/v3/projects/27c58f39e03f42d5862a6a64194f5711/users/04eb412804c247908f8d377a42d1ea63/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.120142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:00.121435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:00.138508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c58f39e03f42d5862a6a64194f5711, user_id=04eb412804c247908f8d377a42d1ea63, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.141498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.162066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 796/1603] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:00 2022] PUT /identity/v3/projects/27c58f39e03f42d5862a6a64194f5711/users/04eb412804c247908f8d377a42d1ea63/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:00.162066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.182984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:00.212466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 808/1604] 10.222.0.30 () {58 vars in 1059 bytes} [Fri Aug 5 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:00.213273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.243264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f580-b539-49b0-8511-a57cc89f8b5e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.248603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.249320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.249604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.249900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.251220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.252373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.263322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 797/1605] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:00.264369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.268084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:00.274857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfdda94-48f4-4ec9-917f-6ab145a86ecd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.282135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.282823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.283128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.283389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] PATH_INFO: `/v3/projects/4b672d5801cb4fdf87d6626f4603e281/users/c7c57f5e11a84bf98d474837adbb0ba7/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.283854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58c0d-00b4-49df-a01b-5abf6946bc64 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.304381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b672d5801cb4fdf87d6626f4603e281, user_id=c7c57f5e11a84bf98d474837adbb0ba7, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.307196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.323543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 798/1606] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:00 2022] PUT /identity/v3/projects/4b672d5801cb4fdf87d6626f4603e281/users/c7c57f5e11a84bf98d474837adbb0ba7/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:00.324231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.341416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.341945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.342227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.343083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.422655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.424288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da98d7-e7d1-4a65-8bde-f4465f6a2862 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.424579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.430223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b10-5e42-41de-a2c9-93ad954fabaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.432108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b10-5e42-41de-a2c9-93ad954fabaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.432470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b10-5e42-41de-a2c9-93ad954fabaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.433133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b10-5e42-41de-a2c9-93ad954fabaf admin admin] PATH_INFO: `/v3/projects/e43f69b71a364a63850d3008309e0c68/users/b89279618557486f9d5b358b654a906e/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.436453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 809/1607] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:00.452444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.456437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0b10-5e42-41de-a2c9-93ad954fabaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43f69b71a364a63850d3008309e0c68, user_id=b89279618557486f9d5b358b654a906e, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.459293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0b10-5e42-41de-a2c9-93ad954fabaf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.469492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38101fe5-1ba1-4f67-931d-617ee9cf3141 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.480328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 799/1608] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:00 2022] PUT /identity/v3/projects/e43f69b71a364a63850d3008309e0c68/users/b89279618557486f9d5b358b654a906e/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:00.481676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.482083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.482365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.503579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:00.504937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:00.540291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.541355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.541645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.541922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] PATH_INFO: `/v3/projects/2373ae73179e4dcba2f284973d6e1814/users/9230732fd72c448a95a0230aad861a1f/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.564132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2373ae73179e4dcba2f284973d6e1814, user_id=9230732fd72c448a95a0230aad861a1f, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.567999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.579472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 800/1609] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:00.579472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.589826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 810/1610] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:00 2022] PUT /identity/v3/projects/2373ae73179e4dcba2f284973d6e1814/users/9230732fd72c448a95a0230aad861a1f/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:00.591473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.604658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:00.650516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:00.674749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f6d49-2436-42f2-9f97-c3f48b2cb08c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.688914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2fe38-b22d-45ba-a6ba-01ab395cb4ed admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:00.693696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92385473-b6e2-44bf-b4ea-f59d79a281c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.694411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92385473-b6e2-44bf-b4ea-f59d79a281c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.694689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92385473-b6e2-44bf-b4ea-f59d79a281c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.694969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92385473-b6e2-44bf-b4ea-f59d79a281c6 admin admin] PATH_INFO: `/v3/projects/368da15828c243f896200dde57aa68f5/users/577c34ab5b6246f09c8ceb53b9296158/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.717971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92385473-b6e2-44bf-b4ea-f59d79a281c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=368da15828c243f896200dde57aa68f5, user_id=577c34ab5b6246f09c8ceb53b9296158, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.722438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92385473-b6e2-44bf-b4ea-f59d79a281c6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.742298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 811/1611] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:00 2022] PUT /identity/v3/projects/368da15828c243f896200dde57aa68f5/users/577c34ab5b6246f09c8ceb53b9296158/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:00.743875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bb69f-1adc-467c-8a87-3aeb5bab29e6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.744193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bb69f-1adc-467c-8a87-3aeb5bab29e6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.744527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bb69f-1adc-467c-8a87-3aeb5bab29e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.768906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-db6bb69f-1adc-467c-8a87-3aeb5bab29e6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:00.771243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db6bb69f-1adc-467c-8a87-3aeb5bab29e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:00.779706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3449c-3f57-4b26-a1b0-a0bccd846e89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:00.780488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3449c-3f57-4b26-a1b0-a0bccd846e89 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.781056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3449c-3f57-4b26-a1b0-a0bccd846e89 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.781056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3449c-3f57-4b26-a1b0-a0bccd846e89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.848132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 812/1612] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:00.849550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.849762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.850019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.873072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:00.874315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:00.883490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 813/1613] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:00.884586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:00.908619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3449c-3f57-4b26-a1b0-a0bccd846e89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:00.910342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3449c-3f57-4b26-a1b0-a0bccd846e89 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:00.921079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 801/1614] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:00.922665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:00.923214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:00.923668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:00.972508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] MFA Rules not processed for user `9230732fd72c448a95a0230aad861a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:00.976411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:01.015996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ae0e5-ec1f-4dcd-8392-f84daf928566 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.029162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:01.034066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071af12b-f0b3-4e04-b85f-ed9076034f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.034802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af12b-f0b3-4e04-b85f-ed9076034f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.035139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af12b-f0b3-4e04-b85f-ed9076034f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.035485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af12b-f0b3-4e04-b85f-ed9076034f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.049260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071af12b-f0b3-4e04-b85f-ed9076034f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.058026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071af12b-f0b3-4e04-b85f-ed9076034f38 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.085562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 814/1615] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:01.088364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fea3e-170f-4f5a-ad0c-df9e8850fce8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.091127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fea3e-170f-4f5a-ad0c-df9e8850fce8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.091490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fea3e-170f-4f5a-ad0c-df9e8850fce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.112050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 802/1616] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:01.112050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.131715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a41bc-b657-4c04-800a-cfcbc128b02f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.140108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.141018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.141382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.144960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.151243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d77fea3e-170f-4f5a-ad0c-df9e8850fce8 None None] MFA Rules not processed for user `577c34ab5b6246f09c8ceb53b9296158`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:01.153025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d77fea3e-170f-4f5a-ad0c-df9e8850fce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:01.162419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.165281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.190792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 803/1617] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:01.192777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.210506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c84b0-8435-4f60-9059-de43adaf39a6 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.221179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.221916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.222287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.222651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.247582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.251071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:01.251744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.274720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 804/1618] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:01.276376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.293666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f93b-c6cf-43b3-8fb5-ea068f4e0863 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.297281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 815/1619] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:01.298763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b273177-51cd-45a8-87d7-5fdee7347516 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.299146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b273177-51cd-45a8-87d7-5fdee7347516 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.299656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b273177-51cd-45a8-87d7-5fdee7347516 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.301262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 816/1620] 10.222.0.30 () {54 vars in 912 bytes} [Fri Aug 5 05:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:01.301947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b273177-51cd-45a8-87d7-5fdee7347516 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.310451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b11810-bb0e-4501-9a67-bd74c4d5a83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.311417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11810-bb0e-4501-9a67-bd74c4d5a83d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.311895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11810-bb0e-4501-9a67-bd74c4d5a83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.312410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b11810-bb0e-4501-9a67-bd74c4d5a83d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.313430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b11810-bb0e-4501-9a67-bd74c4d5a83d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.315518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b11810-bb0e-4501-9a67-bd74c4d5a83d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.337592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b273177-51cd-45a8-87d7-5fdee7347516 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.350188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.352118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.352599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.353503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.355463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.361607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.366709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 805/1621] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:01 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:01.368560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.370479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.370771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.428795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:01.430418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:01.438554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 817/1622] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:01 2022] POST /identity/v3/projects => generated 357 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:01.439468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.519343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e437-cc9a-458e-8e90-1751c49e17cc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.525275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.526079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.526351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.526690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.527529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.529898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.530730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 806/1623] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:13:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:01.531733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.563243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 818/1624] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:01 2022] POST /identity/v3/users => generated 309 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:01.564836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.625604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc17f7-0981-4342-bef1-aebbf5f89175 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.631995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.633140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.633496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.633889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.634993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.639430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.641552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310403f-4240-47cc-bfbd-95c86013b377 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.646970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.648041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.648407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.648776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.671989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 807/1625] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:01 2022] POST /identity/v3/users => generated 285 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:01.673060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.682410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:01.688459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da5bbe-90fa-46f4-9fce-546985cde0ad admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.695841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.696731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.697488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.697967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.698901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.701064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.719301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 808/1626] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:01.719301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.732362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ab5b1-0f05-41c7-9fc9-5082e519026a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.736054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:01.739410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.741263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.741711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.742147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.743048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.745019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.750313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.751951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.767029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 809/1627] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:01.767584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 819/1628] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:01.768534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.779897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.788010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5d62-9b88-42a0-9916-262a18d71cb2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.803911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1268a-f94e-4a29-8a72-b0e12851ae73 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:01.807612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.809403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.809994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.810778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] PATH_INFO: `/v3/projects/0abe61c8675c4f48a020261f98bc29e4/users/7359d3eeabeb457d808fdeef55040b4f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.814832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:01.820282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:01.821639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:01.822093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] PATH_INFO: `/v3/projects/ffd9e9e15b914368b959a786815e2ea6/users/76c149bc0cbe425287397ac5e24534ff/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:01.850547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd9e9e15b914368b959a786815e2ea6, user_id=76c149bc0cbe425287397ac5e24534ff, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.852146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abe61c8675c4f48a020261f98bc29e4, user_id=7359d3eeabeb457d808fdeef55040b4f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:01.853757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.855345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:01.875515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 820/1629] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:01 2022] PUT /identity/v3/projects/ffd9e9e15b914368b959a786815e2ea6/users/76c149bc0cbe425287397ac5e24534ff/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:01.877630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 810/1630] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:01 2022] PUT /identity/v3/projects/0abe61c8675c4f48a020261f98bc29e4/users/7359d3eeabeb457d808fdeef55040b4f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:01.881978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.895147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:01.997607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e60866-3020-449a-adb2-05c256974535 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:02.040322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599dbdd-e1ff-4b96-8dd4-f8fe57584dc7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:02.050109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:02.051062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.051443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.052802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.054082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:02.057223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:02.073205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 811/1631] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:02.077452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:02.098697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:02.099533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.099910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.100278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.101243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:02.102715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62070955-f3a5-49b0-8b11-8d2f083baa23 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:02.103199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:02.112687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f89b8-7509-4128-904b-f4759cbdb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:02.113573 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f89b8-7509-4128-904b-f4759cbdb50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.114381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f89b8-7509-4128-904b-f4759cbdb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.115015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f89b8-7509-4128-904b-f4759cbdb50c admin admin] PATH_INFO: `/v3/projects/0abe61c8675c4f48a020261f98bc29e4/users/7359d3eeabeb457d808fdeef55040b4f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.119416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f89b8-7509-4128-904b-f4759cbdb50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abe61c8675c4f48a020261f98bc29e4, user_id=7359d3eeabeb457d808fdeef55040b4f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:02.122964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 821/1632] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:02.124099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:02.124985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f89b8-7509-4128-904b-f4759cbdb50c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:02.149309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026d96f-82db-480a-8fd0-6e927c0cf5e9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:02.246371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 812/1633] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:02 2022] PUT /identity/v3/projects/0abe61c8675c4f48a020261f98bc29e4/users/7359d3eeabeb457d808fdeef55040b4f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:02.246371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.246371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.246371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.246371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] MFA Rules not processed for user `7359d3eeabeb457d808fdeef55040b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:02.248756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:02.322278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d535b27-aabf-4c55-a9de-0fcb96e001d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:02.323645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d535b27-aabf-4c55-a9de-0fcb96e001d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.324000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d535b27-aabf-4c55-a9de-0fcb96e001d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.324342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d535b27-aabf-4c55-a9de-0fcb96e001d5 admin admin] PATH_INFO: `/v3/projects/ffd9e9e15b914368b959a786815e2ea6/users/76c149bc0cbe425287397ac5e24534ff/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.326747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d535b27-aabf-4c55-a9de-0fcb96e001d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd9e9e15b914368b959a786815e2ea6, user_id=76c149bc0cbe425287397ac5e24534ff, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:02.330040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d535b27-aabf-4c55-a9de-0fcb96e001d5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:02.366788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 822/1634] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:02 2022] PUT /identity/v3/projects/ffd9e9e15b914368b959a786815e2ea6/users/76c149bc0cbe425287397ac5e24534ff/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:02.377183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b1c0d-464c-459b-b1f1-c29c4da18e90 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.377556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b1c0d-464c-459b-b1f1-c29c4da18e90 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.377899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b1c0d-464c-459b-b1f1-c29c4da18e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.406062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:02.465407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 813/1635] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:02.472732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:02.484540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-963b1c0d-464c-459b-b1f1-c29c4da18e90 None None] MFA Rules not processed for user `76c149bc0cbe425287397ac5e24534ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:02.486186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-963b1c0d-464c-459b-b1f1-c29c4da18e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:02.568188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:02.634695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 823/1636] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:02.639560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fdd11-c248-4181-83b3-615baa63c6b3 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.640090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fdd11-c248-4181-83b3-615baa63c6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.640407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fdd11-c248-4181-83b3-615baa63c6b3 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.642933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 824/1637] 10.222.0.30 () {54 vars in 962 bytes} [Fri Aug 5 05:13:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:02.652689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877e89a-6ed1-4ddf-84af-abebf3115c6e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.653231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877e89a-6ed1-4ddf-84af-abebf3115c6e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.655537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877e89a-6ed1-4ddf-84af-abebf3115c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.659376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba7620-e064-4635-bb50-644572c0c6ff None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:02.688184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8877e89a-6ed1-4ddf-84af-abebf3115c6e None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:02.690136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8877e89a-6ed1-4ddf-84af-abebf3115c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:02.756622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:02.757330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.757623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.758073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.805776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 825/1638] 10.222.0.30 () {58 vars in 1059 bytes} [Fri Aug 5 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:02.819516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53756bf1-6cfa-4c33-8fd8-29ca33fdd6d0 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.820277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53756bf1-6cfa-4c33-8fd8-29ca33fdd6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.820277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53756bf1-6cfa-4c33-8fd8-29ca33fdd6d0 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.821617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 826/1639] 10.222.0.30 () {54 vars in 912 bytes} [Fri Aug 5 05:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:13:02.831464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.832063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.832539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:02.876042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:02.876450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:02.877945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:02.878439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:02.899796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 827/1640] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:02.900095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 814/1641] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:02.966525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:02.982257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cea387-19cd-43ca-8919-d8bcc0341f79 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:02.987929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:02.988854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:02.989185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:02.990154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:03.026211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:03.027999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:03.042917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 828/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:13:03.103550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:03.114646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d22b-fd7b-4eb5-877e-524ec157c8f5 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:03.128477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:03.129246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:03.129547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:03.129859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:03.155421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:03.168029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:03.212632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 815/1643] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:03.528796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:03.550118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ff239-bc90-40b9-b559-37860a45a5f1 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:03.565271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7669601-48b6-4163-8a46-621afa0c371a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:03.565991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669601-48b6-4163-8a46-621afa0c371a service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:03.566288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669601-48b6-4163-8a46-621afa0c371a service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:03.566541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669601-48b6-4163-8a46-621afa0c371a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:03.683334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7669601-48b6-4163-8a46-621afa0c371a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:03.685058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7669601-48b6-4163-8a46-621afa0c371a service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:03.703105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:03.714645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 829/1644] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:03.889236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:04.086549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab3020-e146-4042-8879-70d9800fd1f0 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:04.097922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:04.099444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:04.100013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:04.100461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] PATH_INFO: `/v3/users/9230732fd72c448a95a0230aad861a1f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:04.102333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9230732fd72c448a95a0230aad861a1f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:04.104308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: WARNING py.warnings [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:13:04.104308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: warnings.warn(msg) Aug 05 05:13:04.104308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: Aug 05 05:13:04.106041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:04.151418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:04.223165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9230732fd72c448a95a0230aad861a1f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:04.223972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9230732fd72c448a95a0230aad861a1f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:04.254114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9230732fd72c448a95a0230aad861a1f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:04.256060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9230732fd72c448a95a0230aad861a1f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:04.273696 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9230732fd72c448a95a0230aad861a1f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:04.336210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 816/1645] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:03 2022] DELETE /identity/v3/users/9230732fd72c448a95a0230aad861a1f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:04.340865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:04.341349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:04.341734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:04.365036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:04.366309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:04.453647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 830/1646] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:04.493039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:04.504749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:04.675238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b97698-3657-40de-997e-0250d8fe6761 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:04.682366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec884df0-503b-466b-bed8-8d6a55f402e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:04.683365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884df0-503b-466b-bed8-8d6a55f402e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:04.684368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884df0-503b-466b-bed8-8d6a55f402e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:04.684889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884df0-503b-466b-bed8-8d6a55f402e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:04.698486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec884df0-503b-466b-bed8-8d6a55f402e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:04.700491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec884df0-503b-466b-bed8-8d6a55f402e5 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:04.717223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 817/1647] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:05.155017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:05.155017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:05.155017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbc8bf-87de-4236-b0c4-acdd13620ff9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:05.158904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:05.158904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:05.158904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:05.158904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] PATH_INFO: `/v3/users/0c27df6b5e8042e2be0092d727a2344f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:05.160323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c27df6b5e8042e2be0092d727a2344f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:05.160323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:05.240126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c27df6b5e8042e2be0092d727a2344f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.241302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c27df6b5e8042e2be0092d727a2344f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.254190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c27df6b5e8042e2be0092d727a2344f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.255322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c27df6b5e8042e2be0092d727a2344f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.271683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c27df6b5e8042e2be0092d727a2344f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.311792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:05.313442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:05.314051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:05.332806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 831/1648] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:05 2022] DELETE /identity/v3/users/0c27df6b5e8042e2be0092d727a2344f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:05.391897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:05.393415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:05.564089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 818/1649] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:05.612130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:05.615475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:05.699622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:05.701698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:05.732776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22a1bb-0893-42d4-8739-6303db2d240e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:05.753023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:05.753710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:05.753983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:05.754413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] PATH_INFO: `/v3/projects/2373ae73179e4dcba2f284973d6e1814` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:05.757463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2373ae73179e4dcba2f284973d6e1814)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:05.759393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:05.763577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17822b4-af72-4ed9-abaf-26d6e1a65fa0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:05.774017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:05.776307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:05.776866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:05.777329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:05.804774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:05.807912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:05.823272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 832/1650] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:05.825686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:05.843673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2373ae73179e4dcba2f284973d6e1814', 'request_id': 'req-60d7a5cf-ef2f-4094-b018-df6af8992136'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.854624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2373ae73179e4dcba2f284973d6e1814', 'request_id': 'req-60d7a5cf-ef2f-4094-b018-df6af8992136'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.864329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2373ae73179e4dcba2f284973d6e1814', 'request_id': 'req-60d7a5cf-ef2f-4094-b018-df6af8992136'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.892446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] The token cache is being invalidate because project 2373ae73179e4dcba2f284973d6e1814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:05.893196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.894302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:05.897273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 819/1651] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:05 2022] DELETE /identity/v3/projects/2373ae73179e4dcba2f284973d6e1814 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:05.901083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:05.915151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64952c20-4743-43c2-8b02-139328b6530c service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:05.935380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:06.021808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:06.022751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:06.023097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:06.023471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] PATH_INFO: `/v3/users/7359d3eeabeb457d808fdeef55040b4f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:06.025761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7359d3eeabeb457d808fdeef55040b4f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:06.028734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:06.116932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7359d3eeabeb457d808fdeef55040b4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:06.117966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7359d3eeabeb457d808fdeef55040b4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:06.127321 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7359d3eeabeb457d808fdeef55040b4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:06.128402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7359d3eeabeb457d808fdeef55040b4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:06.143218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-a9694e5e-7a90-48ec-85bb-556784859a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7359d3eeabeb457d808fdeef55040b4f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:06.180440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 833/1652] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:05 2022] DELETE /identity/v3/users/7359d3eeabeb457d808fdeef55040b4f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:08.093901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:08.095551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:08.122401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a5cf-ef2f-4094-b018-df6af8992136 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:08.279891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:08.281257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:08.281614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:08.282729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] PATH_INFO: `/v3/projects/d55cdc16c75f4fda92582251296ae60c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:08.286435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55cdc16c75f4fda92582251296ae60c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:08.289691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:08.432773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55cdc16c75f4fda92582251296ae60c', 'request_id': 'req-05801f05-c505-4d86-8310-04a1f3178c5f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:08.450038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55cdc16c75f4fda92582251296ae60c', 'request_id': 'req-05801f05-c505-4d86-8310-04a1f3178c5f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:08.470649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55cdc16c75f4fda92582251296ae60c', 'request_id': 'req-05801f05-c505-4d86-8310-04a1f3178c5f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:08.498894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] The token cache is being invalidate because project d55cdc16c75f4fda92582251296ae60c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:08.499188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:08.500736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05801f05-c505-4d86-8310-04a1f3178c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:08.505771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 820/1653] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:08 2022] DELETE /identity/v3/projects/d55cdc16c75f4fda92582251296ae60c => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:08.511019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:08.513631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:08.515429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:08.599243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:08.608726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:08.829663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 834/1654] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:08.837851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:08.838175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:08.838435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:08.937582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:08.938781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:08.964448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 821/1655] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:08.965938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:08.982325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e29ee1-77ab-4967-a23c-7512989062d1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.001197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.001635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.001994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.002775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.003165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:09.003473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.004156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.007744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.014810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:09.042549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837bcf8-4a64-4d1d-a0f4-a2044be71098 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.047696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 835/1656] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:09.053356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.067573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1013c-31b7-480d-84da-fe009a6d56f6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.071307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.072205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.072432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.074041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] PATH_INFO: `/v3/projects/0abe61c8675c4f48a020261f98bc29e4` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.075584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0abe61c8675c4f48a020261f98bc29e4)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.077752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.080361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.080850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.081195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.081565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.082366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.084653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.140326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:09.153714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 836/1657] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:09 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:09.154938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.244620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:09.259811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0abe61c8675c4f48a020261f98bc29e4', 'request_id': 'req-379a4207-c6de-4af4-b703-31dc40fe0567'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:09.282455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0abe61c8675c4f48a020261f98bc29e4', 'request_id': 'req-379a4207-c6de-4af4-b703-31dc40fe0567'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:09.295106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866a723-1161-4b8b-9aea-46c5cf6b967d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.304621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0abe61c8675c4f48a020261f98bc29e4', 'request_id': 'req-379a4207-c6de-4af4-b703-31dc40fe0567'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:09.363919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] The token cache is being invalidate because project 0abe61c8675c4f48a020261f98bc29e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:09.364168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:09.366564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379a4207-c6de-4af4-b703-31dc40fe0567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:09.368449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 822/1658] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:09 2022] DELETE /identity/v3/projects/0abe61c8675c4f48a020261f98bc29e4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:09.374281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4724aa2-57a7-4608-b402-542446dc6985 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.374479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4724aa2-57a7-4608-b402-542446dc6985 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.374764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4724aa2-57a7-4608-b402-542446dc6985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.438756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e4724aa2-57a7-4608-b402-542446dc6985 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:09.441119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e4724aa2-57a7-4608-b402-542446dc6985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:09.442355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.443010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.443288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.443560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.444421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.446771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.464112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 823/1659] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:09.470201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.470963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.471382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.496928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:09.498326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:09.504129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 837/1660] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:09 2022] POST /identity/v3/users => generated 283 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:09.511284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.517917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:09.524242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 824/1661] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:09.528832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.551533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e77e4f-b4ea-4cd1-b653-afbeb683eafd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.558318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.570993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc38dfc-77b0-4bdd-910c-bfb5d73f23ce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.579265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.580644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.580936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.582173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:09.584349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.589778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.596664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.597311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.597588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.598674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.599401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.601089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.622256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 825/1662] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:09.626520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.651422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 838/1663] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:09.655289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.657107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bdc14-24d1-4000-8370-79a1a885f265 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.682514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3428099-f186-4685-a4df-4a92350d3410 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:09.685039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.685684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.685962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.686235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.687481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.689466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.713660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:09.714320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:09.714603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:09.714886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:09.717895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:09.719803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:09.774046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 839/1664] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:09.777073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.811193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 826/1665] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:09 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:09.812108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:09.829727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49073169-c260-40d5-9cc8-823c45ec4f2f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.044813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98961c90-4321-44cf-9d80-a0d0af935120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.049725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98961c90-4321-44cf-9d80-a0d0af935120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.050053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98961c90-4321-44cf-9d80-a0d0af935120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.050335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98961c90-4321-44cf-9d80-a0d0af935120 admin admin] PATH_INFO: `/v3/projects/cb67419a21c94f158aac708300604332/users/0077cf68960b46dda92fe21b2465ab3c/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.076279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acb3d9-29d6-4afc-a2dd-cd44f22919da admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.081574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.082201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.082488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.083066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.083748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.086015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.109342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98961c90-4321-44cf-9d80-a0d0af935120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb67419a21c94f158aac708300604332, user_id=0077cf68960b46dda92fe21b2465ab3c, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.123881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98961c90-4321-44cf-9d80-a0d0af935120 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.159803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 827/1666] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:09 2022] POST /identity/v3/users => generated 287 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:10.163798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:10.165114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.182661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 840/1667] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:09 2022] PUT /identity/v3/projects/cb67419a21c94f158aac708300604332/users/0077cf68960b46dda92fe21b2465ab3c/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:10.194889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4568efd-2571-470e-9211-dbacdd67f27d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.195036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.195036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.195036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.330570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] MFA Rules not processed for user `0077cf68960b46dda92fe21b2465ab3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:10.354741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:10.367070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.367732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.368039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.368321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.369038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.375151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.401693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 828/1668] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:10.406718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.435009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719722b-03af-4205-a55b-23dcbc5b0097 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.448254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.448997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.449288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.449651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.451930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.453606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.458846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:10.490182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 829/1669] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:10.497266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.520263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 841/1670] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:10.525941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.531491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fd7b9-9858-48dd-9eb7-37b2acf4c69d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.547279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd077e64-c467-4872-8715-0f33413a9f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.548034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd077e64-c467-4872-8715-0f33413a9f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.548375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd077e64-c467-4872-8715-0f33413a9f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.548662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd077e64-c467-4872-8715-0f33413a9f21 admin admin] PATH_INFO: `/v3/projects/c74c162641f94714aedb76d4b07585b1/users/188226af2d2c463ebae7afd8d9caf4c1/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.557811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ecf22-c0dd-46c2-9164-e4be3ba4220e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.568358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.569112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.570367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.571721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.573595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.579360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.597741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd077e64-c467-4872-8715-0f33413a9f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74c162641f94714aedb76d4b07585b1, user_id=188226af2d2c463ebae7afd8d9caf4c1, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.600968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd077e64-c467-4872-8715-0f33413a9f21 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.636028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 830/1671] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:10 2022] PUT /identity/v3/projects/c74c162641f94714aedb76d4b07585b1/users/188226af2d2c463ebae7afd8d9caf4c1/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:10.649840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.651402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.651402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.655590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 842/1672] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:10 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:10.660508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.723364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] MFA Rules not processed for user `188226af2d2c463ebae7afd8d9caf4c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:10.724637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:10.800594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a22621-eb2f-44c1-9f1c-95045a7903b1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.815598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.816825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.816825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.817322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.817828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.819882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.866591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 843/1673] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:10 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:10.870816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.883636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 831/1674] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:10.887858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.895328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda3009-e6aa-4a9c-97b9-16e44af5bd5b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.908181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.909118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.909118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.909381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.910082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.911019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93497f12-4bc4-4cf4-86e2-4db29be81fb6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.913847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.926216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.926899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.927183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.927458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:10.928164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:10.930414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:10.936863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 844/1675] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:10.947560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:10.974521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade1f8b-4756-46d0-99fc-623064e2085a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:10.985124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:10.985839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:10.986124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:10.986405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] PATH_INFO: `/v3/projects/0f0259cdb1e94a08b91c41aadd43d7c2/users/ee714c90af564a7881a7d5e27ca0d4f0/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.001904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 832/1676] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:10 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:11.005958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:11.043895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0259cdb1e94a08b91c41aadd43d7c2, user_id=ee714c90af564a7881a7d5e27ca0d4f0, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:11.046649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:11.077754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 845/1677] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:10 2022] PUT /identity/v3/projects/0f0259cdb1e94a08b91c41aadd43d7c2/users/ee714c90af564a7881a7d5e27ca0d4f0/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:11.081907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:11.125215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:11.168806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a24c75-d53c-4e18-8194-a66513c75796 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:11.214270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd0d43-912a-4ca8-96ff-d5c10e48c19d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:11.222843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:11.223562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.223876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.224158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.224877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:11.226564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:11.249872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 846/1678] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:11.254008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:11.265298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:11.272000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02a66-751c-4441-9751-be2d37bae36b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:11.288530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:11.289167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.289452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.289723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.290412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:11.290708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17db93-2024-46ad-93ed-1be1f33797f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:11.291806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17db93-2024-46ad-93ed-1be1f33797f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.292097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17db93-2024-46ad-93ed-1be1f33797f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.292535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17db93-2024-46ad-93ed-1be1f33797f7 admin admin] PATH_INFO: `/v3/projects/0f0259cdb1e94a08b91c41aadd43d7c2/users/ee714c90af564a7881a7d5e27ca0d4f0/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.294538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17db93-2024-46ad-93ed-1be1f33797f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0259cdb1e94a08b91c41aadd43d7c2, user_id=ee714c90af564a7881a7d5e27ca0d4f0, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:11.297171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17db93-2024-46ad-93ed-1be1f33797f7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:11.300067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:11.336181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 847/1679] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:11 2022] PUT /identity/v3/projects/0f0259cdb1e94a08b91c41aadd43d7c2/users/ee714c90af564a7881a7d5e27ca0d4f0/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:11.348381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.348674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.349033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.367478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 833/1680] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:11 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:11.368500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:11.414660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85ed11-91cd-488a-a3f4-3760ed31a9fb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:11.423204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] MFA Rules not processed for user `ee714c90af564a7881a7d5e27ca0d4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:11.424916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:11.675366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1c9c5-9876-4acc-88b1-30b1b0ba467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:11.677419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c9c5-9876-4acc-88b1-30b1b0ba467b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.678318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c9c5-9876-4acc-88b1-30b1b0ba467b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.679124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c9c5-9876-4acc-88b1-30b1b0ba467b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.680910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c9c5-9876-4acc-88b1-30b1b0ba467b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:11.685915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c9c5-9876-4acc-88b1-30b1b0ba467b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:11.699974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 848/1681] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:11.732331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 834/1682] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:11.736696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:11.752768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.755366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.757014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.770324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0ec9c-8b2a-4ee0-88a8-aafe49c885dd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:11.809830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:11.812796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:11.813024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:11.813306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] PATH_INFO: `/v3/projects/901f6b595ad241ad9276489a2769ded1/users/fdf588b1ca53416cbd3d352aa8397e8d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:11.863094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901f6b595ad241ad9276489a2769ded1, user_id=fdf588b1ca53416cbd3d352aa8397e8d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:11.874050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:11.876372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] MFA Rules not processed for user `04eb412804c247908f8d377a42d1ea63`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:11.882009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:11.919993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 849/1683] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:11 2022] PUT /identity/v3/projects/901f6b595ad241ad9276489a2769ded1/users/fdf588b1ca53416cbd3d352aa8397e8d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:11.921217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:11.923759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:12.066007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baaa77d-c425-45b5-bdab-e0772eddf0e2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:12.080236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:12.081037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.081329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.081594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.099731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 835/1684] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:12.101867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:12.124412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9da346-7f4f-4fee-a158-5dfe35108c32 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:12.131505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:12.241726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:12.243210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:12.262611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:12.263252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.263574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.263818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.264529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:12.267895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 850/1685] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:12.269593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:12.269708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:12.272113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:12.281046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:12.307893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 836/1686] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:12.312027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:12.336248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7baa13-711c-4edb-a1d3-12b80b04dabd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:12.360548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bb358-c53b-40f5-bc0b-aa4390c11492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:12.361247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bb358-c53b-40f5-bc0b-aa4390c11492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.361532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bb358-c53b-40f5-bc0b-aa4390c11492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.361811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bb358-c53b-40f5-bc0b-aa4390c11492 admin admin] PATH_INFO: `/v3/projects/901f6b595ad241ad9276489a2769ded1/users/fdf588b1ca53416cbd3d352aa8397e8d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.372637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bb358-c53b-40f5-bc0b-aa4390c11492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901f6b595ad241ad9276489a2769ded1, user_id=fdf588b1ca53416cbd3d352aa8397e8d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:12.375398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bb358-c53b-40f5-bc0b-aa4390c11492 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:12.413541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 837/1687] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:12 2022] PUT /identity/v3/projects/901f6b595ad241ad9276489a2769ded1/users/fdf588b1ca53416cbd3d352aa8397e8d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:12.428585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.430757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.431439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.498235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] MFA Rules not processed for user `fdf588b1ca53416cbd3d352aa8397e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:12.499473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:12.504911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a804-b0dd-423b-aeb8-40ad0788c043 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:12.653633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 838/1688] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:12.656405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:12.657178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.657583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.657916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.658727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:12.735934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09a87f-9a18-496c-8358-4f5fb1dc2651 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:12.740180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:12.740890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.741171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.741446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.771017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:12.790707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:12.793312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:12.832978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 851/1689] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:12.835080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:12.836968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:12.853066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda83a5-c9fc-4c89-9c26-891b64a8a495 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:12.859156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:12.860527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:12.860933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:12.861347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:12.861763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:12.862044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:12.874691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:12.877449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:12.882428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 839/1690] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:12.893094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 852/1691] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:13.104563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:13.106067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:13.200384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:13.279110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:13.281782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa401cb-2095-4801-ac39-0a3e7822d27b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:13.303931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:13.303931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:13.303931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:13.303931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] PATH_INFO: `/v3/users/16d408692a9f4b689c0a12e03e0f8e05` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:13.304217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d408692a9f4b689c0a12e03e0f8e05)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:13.304288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382104eb-9d6e-4a2b-b335-59dcc322ad90 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:13.305003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:13.320312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:13.320958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:13.321239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:13.321511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:13.402947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d408692a9f4b689c0a12e03e0f8e05'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:13.404127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d408692a9f4b689c0a12e03e0f8e05'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:13.420234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d408692a9f4b689c0a12e03e0f8e05'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:13.421520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d408692a9f4b689c0a12e03e0f8e05'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:13.440661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6823f7d4-29e2-4f44-bc6c-0485d627df69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d408692a9f4b689c0a12e03e0f8e05'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:13.476357 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 840/1692] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:13 2022] DELETE /identity/v3/users/16d408692a9f4b689c0a12e03e0f8e05 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:13.482314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:13.482602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:13.482878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:13.525047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:13.526298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:13.541913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:13.543407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:13.586554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 853/1693] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:13:13.719874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 841/1694] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:13.792996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:13.794508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:14.091132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452baf3d-b532-42fe-b70c-bdd0d6567bff service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:14.199320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d25e2-ff76-418f-ad7c-80039d5206f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:14.201274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d25e2-ff76-418f-ad7c-80039d5206f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:14.201792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d25e2-ff76-418f-ad7c-80039d5206f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:14.202079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d25e2-ff76-418f-ad7c-80039d5206f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:14.255806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d25e2-ff76-418f-ad7c-80039d5206f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:14.269553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d25e2-ff76-418f-ad7c-80039d5206f1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:14.283627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 854/1695] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:16.210859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:16.215110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:16.246128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ecda6-89cb-40e9-826d-41a6e0b0ff74 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:16.276342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:16.286763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:16.287241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:16.287757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] PATH_INFO: `/v3/projects/bdb0ab67f8e54568a8be4a7d80411863` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:16.297894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb0ab67f8e54568a8be4a7d80411863)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:16.301138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:16.471029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:16.484132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb0ab67f8e54568a8be4a7d80411863', 'request_id': 'req-93ef24e1-9e02-4fd2-aca8-649f28d24550'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:16.505566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb0ab67f8e54568a8be4a7d80411863', 'request_id': 'req-93ef24e1-9e02-4fd2-aca8-649f28d24550'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:16.517926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb0ab67f8e54568a8be4a7d80411863', 'request_id': 'req-93ef24e1-9e02-4fd2-aca8-649f28d24550'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:16.546402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] The token cache is being invalidate because project bdb0ab67f8e54568a8be4a7d80411863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:16.547063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:16.548455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-93ef24e1-9e02-4fd2-aca8-649f28d24550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:16.550622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 842/1696] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:16 2022] DELETE /identity/v3/projects/bdb0ab67f8e54568a8be4a7d80411863 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:16.555420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:16.556441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:16.556920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:16.596053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:16.600344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:16.676829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:16.760238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 855/1697] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:16.769554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:16.770509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:16.772629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:16.843750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:16.847276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:16.868896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 843/1698] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:16.873376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:16.890836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099e76f-8d05-410f-85a4-c6c9d654db93 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:16.904405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:16.908350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:16.908936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:16.911547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:16.913690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:16.914248 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:16.919646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:16.953096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 856/1699] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:16.957323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:16.970791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a5fc2-b05a-43dd-b5a1-0ee3d4fafdd6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:16.984393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:16.985275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:16.985805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:16.985906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:16.986584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:16.993567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:17.095323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 844/1700] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:16 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:17.096197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:17.120695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d89b4-5fe8-452e-adef-d0d670bf0dcf admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:17.303759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:17.305879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:17.306481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:17.306923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:17.308084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:17.315036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:17.394932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 857/1701] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:17 2022] POST /identity/v3/users => generated 290 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:17.396422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:17.409047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f8937-56ab-49ac-9f03-d067c87842e1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:17.422297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:17.422749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:17.423034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:17.423311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:17.424411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:17.425694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:17.458481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 845/1702] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:17.459846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:17.472463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade792fe-2b84-4787-955b-96474e8e4b7e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:17.485295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21968af-e7a6-4f81-af90-367cb0661674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:17.485939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21968af-e7a6-4f81-af90-367cb0661674 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:17.486225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21968af-e7a6-4f81-af90-367cb0661674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:17.486502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21968af-e7a6-4f81-af90-367cb0661674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:17.487216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21968af-e7a6-4f81-af90-367cb0661674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:17.488924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21968af-e7a6-4f81-af90-367cb0661674 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:17.524808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 858/1703] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:17.529170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:17.551945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ece097-3741-4d02-b968-44e998bfeac3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:17.563631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:17.581094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:17.581560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:17.581843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] PATH_INFO: `/v3/projects/15b2bc917952432b8c538f07bbb6729d/users/5e907263e7ad4337b7e7502c42009468/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:17.625931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b2bc917952432b8c538f07bbb6729d, user_id=5e907263e7ad4337b7e7502c42009468, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:17.633931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:17.679008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 846/1704] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:17 2022] PUT /identity/v3/projects/15b2bc917952432b8c538f07bbb6729d/users/5e907263e7ad4337b7e7502c42009468/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:17.690860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b0c6d-6464-46fe-b89b-530833ba5d31 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:17.691346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b0c6d-6464-46fe-b89b-530833ba5d31 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:17.691346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b0c6d-6464-46fe-b89b-530833ba5d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:17.800758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bd2b0c6d-6464-46fe-b89b-530833ba5d31 None None] MFA Rules not processed for user `5e907263e7ad4337b7e7502c42009468`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:17.804661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bd2b0c6d-6464-46fe-b89b-530833ba5d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:17.954284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 859/1705] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:17.955000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:18.116485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540a536-88cb-4578-95e0-c31500b5b20e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:18.129347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:18.130430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.131310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.132371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.133815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:18.137125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:18.172082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc78462-64bb-485b-b9e4-31eb892ed98a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.172336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc78462-64bb-485b-b9e4-31eb892ed98a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.172725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc78462-64bb-485b-b9e4-31eb892ed98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.210436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 847/1706] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:17 2022] POST /identity/v3/projects => generated 381 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:18.213715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:18.239728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0bc78462-64bb-485b-b9e4-31eb892ed98a None None] MFA Rules not processed for user `c7c57f5e11a84bf98d474837adbb0ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:18.241633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0bc78462-64bb-485b-b9e4-31eb892ed98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:18.289098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:18.383033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 860/1707] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:18.383662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.384107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.384479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.396387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:18.431044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36f0dc-ce85-4215-bfc4-73a3614d9ad8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:18.444206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e9a8b-04a1-478d-aecd-38b7e25515e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:18.444206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e9a8b-04a1-478d-aecd-38b7e25515e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.444206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e9a8b-04a1-478d-aecd-38b7e25515e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.444206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e9a8b-04a1-478d-aecd-38b7e25515e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.444649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e9a8b-04a1-478d-aecd-38b7e25515e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:18.444649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e9a8b-04a1-478d-aecd-38b7e25515e5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:18.460318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] MFA Rules not processed for user `28432f2e1dea486cb163fa2dab6e443f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:18.462206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:18.488272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 848/1708] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:18 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:18.490227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.490227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.490688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.530122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] MFA Rules not processed for user `c7c57f5e11a84bf98d474837adbb0ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:18.531492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:18.540392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 849/1709] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:18.541844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:18.555616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ff64a-1623-4ae9-a215-813c89d3984f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:18.561002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:18.561641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.562004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.562288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.563540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:18.565509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:18.592993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 850/1710] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:18.592993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:18.594400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:18.602477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 861/1711] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:18.603075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:18.631794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd364-4c61-40c6-9c17-9ac3b3e0dc96 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:18.648766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf22e58-0334-4f43-b535-52054ebda8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:18.653603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf22e58-0334-4f43-b535-52054ebda8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.653922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf22e58-0334-4f43-b535-52054ebda8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.654201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf22e58-0334-4f43-b535-52054ebda8a7 admin admin] PATH_INFO: `/v3/projects/350f89ff304a457281b466344d58d27a/users/824b8a40bccd4749826c7e535da039b3/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.689929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf22e58-0334-4f43-b535-52054ebda8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350f89ff304a457281b466344d58d27a, user_id=824b8a40bccd4749826c7e535da039b3, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:18.692483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf22e58-0334-4f43-b535-52054ebda8a7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:18.721064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 862/1712] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:18 2022] PUT /identity/v3/projects/350f89ff304a457281b466344d58d27a/users/824b8a40bccd4749826c7e535da039b3/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:18.722433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.722719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.725619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.773900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829ad00-6913-4820-9006-bd3f173daad9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:18.778894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] MFA Rules not processed for user `28432f2e1dea486cb163fa2dab6e443f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:18.780503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:18.909693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 863/1713] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:18.910765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:18.914072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc5ef2-a463-43f8-9e1b-0f11030e4b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:18.915497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc5ef2-a463-43f8-9e1b-0f11030e4b47 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:18.916341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc5ef2-a463-43f8-9e1b-0f11030e4b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:18.917553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc5ef2-a463-43f8-9e1b-0f11030e4b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:18.982140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:19.069052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f303d-1d1f-4fe5-a73a-f2f1d9b74b47 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:19.076403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:19.077143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.077500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.077841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.078632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:19.080344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:19.122130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc5ef2-a463-43f8-9e1b-0f11030e4b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:19.122130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc5ef2-a463-43f8-9e1b-0f11030e4b47 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:19.122873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:19.130776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 864/1714] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:19.132002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:19.135102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:19.138955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 851/1715] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:19.141745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.142039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.142462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.153067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d6dc-1564-4e55-95e8-3b03538b026e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:19.159038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:19.159876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.160234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.160650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.167978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] MFA Rules not processed for user `04eb412804c247908f8d377a42d1ea63`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:19.169641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:19.173731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:19.176088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:19.187144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 865/1716] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:19.192897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:19.210885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b88d7-b83a-4f40-bc43-bd08bd175f4d service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:19.220479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b069e8-b21a-4fa7-aa82-3845689e5c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:19.223697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b069e8-b21a-4fa7-aa82-3845689e5c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.224012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b069e8-b21a-4fa7-aa82-3845689e5c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.224297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b069e8-b21a-4fa7-aa82-3845689e5c4d admin admin] PATH_INFO: `/v3/projects/350f89ff304a457281b466344d58d27a/users/824b8a40bccd4749826c7e535da039b3/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.230586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b069e8-b21a-4fa7-aa82-3845689e5c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=350f89ff304a457281b466344d58d27a, user_id=824b8a40bccd4749826c7e535da039b3, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:19.234941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b069e8-b21a-4fa7-aa82-3845689e5c4d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:19.286150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 866/1717] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:19 2022] PUT /identity/v3/projects/350f89ff304a457281b466344d58d27a/users/824b8a40bccd4749826c7e535da039b3/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:19.297550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.297800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.298101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.313784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 852/1718] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:19.352302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] MFA Rules not processed for user `824b8a40bccd4749826c7e535da039b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:19.354617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:19.452372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 867/1719] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:19.467653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:19.469361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:19.567021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296c8c-eb71-4ca7-be90-96e49de7d9c1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:19.624281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f602188-899b-4f99-aad1-e2b4fa059a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:19.625221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f602188-899b-4f99-aad1-e2b4fa059a21 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.625621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f602188-899b-4f99-aad1-e2b4fa059a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.626008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f602188-899b-4f99-aad1-e2b4fa059a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.637511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f602188-899b-4f99-aad1-e2b4fa059a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:19.639131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f602188-899b-4f99-aad1-e2b4fa059a21 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:19.650299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 853/1720] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:19.794734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:19.809556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b2f3a-3d98-4362-93d2-d3b4f70bb624 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:19.827908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafbfe8-a1a1-4399-8673-b5ceae1cbee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:19.828788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafbfe8-a1a1-4399-8673-b5ceae1cbee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:19.829229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafbfe8-a1a1-4399-8673-b5ceae1cbee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:19.829508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafbfe8-a1a1-4399-8673-b5ceae1cbee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:19.931268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafbfe8-a1a1-4399-8673-b5ceae1cbee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:19.936402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafbfe8-a1a1-4399-8673-b5ceae1cbee6 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:19.968404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 868/1721] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:20.175958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bec2-2ea8-4da3-8722-612e5b63dfb5 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.176828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bec2-2ea8-4da3-8722-612e5b63dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.177523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bec2-2ea8-4da3-8722-612e5b63dfb5 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.183468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 854/1722] 10.222.0.30 () {54 vars in 912 bytes} [Fri Aug 5 05:13:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:20.188736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4f92e-03b3-4733-be0c-6970c0c9d75e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.189034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4f92e-03b3-4733-be0c-6970c0c9d75e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.189307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4f92e-03b3-4733-be0c-6970c0c9d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.213278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73062813-421d-48ee-a2b7-c909d839bacf None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.213767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73062813-421d-48ee-a2b7-c909d839bacf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.214388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73062813-421d-48ee-a2b7-c909d839bacf None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.216898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 855/1723] 10.222.0.30 () {54 vars in 912 bytes} [Fri Aug 5 05:13:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:13:20.226666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.227329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.227869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.279309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-17d4f92e-03b3-4733-be0c-6970c0c9d75e None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:20.284283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-17d4f92e-03b3-4733-be0c-6970c0c9d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:20.289804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:20.318652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:20.325513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:20.401720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 869/1724] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:13:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:20.404069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.405621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.406097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.431365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 856/1725] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:13:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:20.432677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:20.434271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:20.448201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:20.452347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] MFA Rules not processed for user `b89279618557486f9d5b358b654a906e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:20.452521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359747-c064-4ea7-8c11-3e669ac42bb5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:20.454858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:20.456836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:20.458783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.459132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.459419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.469415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:20.470971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:20.485351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 857/1726] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:20.505384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:20.535708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf8cae-0efc-43d3-9efe-6468dc81c15b service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:20.545695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98e682-4ad9-47d1-94d0-894ce724acb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:20.547714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98e682-4ad9-47d1-94d0-894ce724acb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:20.548668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98e682-4ad9-47d1-94d0-894ce724acb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:20.549718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98e682-4ad9-47d1-94d0-894ce724acb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:20.573098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98e682-4ad9-47d1-94d0-894ce724acb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:20.575605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98e682-4ad9-47d1-94d0-894ce724acb6 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:20.615563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 870/1727] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:20.625338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 858/1728] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:21.533468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:21.572306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5219c8-751b-48f6-8378-9d9cdfc09a70 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:21.573272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5219c8-751b-48f6-8378-9d9cdfc09a70 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:21.573931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5219c8-751b-48f6-8378-9d9cdfc09a70 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:21.575685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 859/1729] 10.222.0.30 () {54 vars in 961 bytes} [Fri Aug 5 05:13:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:21.591359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:21.591639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:21.592121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:21.630998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:21.633153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:21.691202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdc1a8-5625-4871-a4d7-c4867fc512e0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:21.713201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:21.713868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:21.714156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:21.714436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:21.762579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:21.766549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:21.803751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 860/1730] 10.222.0.30 () {58 vars in 1058 bytes} [Fri Aug 5 05:13:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:21.805681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:21.810309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 871/1731] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:21.812757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:21.816382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:21.847521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad0d29-853d-4ac5-951f-4ed088da79fc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:21.854527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd255c-b300-43eb-9e90-c0182d103cd7 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:21.862780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:21.868773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:21.869467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:21.872268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:21.872553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:21.875489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:21.875947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:21.876301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:21.876556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:21.879869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:21.906077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:21.908856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:21.910368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d71af-aa93-41ac-8311-679bb7917740 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:21.910932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadc9b1-b8c3-4ea6-9540-1abf11a75821 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:21.931967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 861/1732] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:21.956695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 872/1733] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:22.839822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aceddb1-ffb8-456b-ad7d-68836088b4cc None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:22.840379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aceddb1-ffb8-456b-ad7d-68836088b4cc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:22.840379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aceddb1-ffb8-456b-ad7d-68836088b4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:22.884616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2aceddb1-ffb8-456b-ad7d-68836088b4cc None None] MFA Rules not processed for user `c7c57f5e11a84bf98d474837adbb0ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:22.887571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2aceddb1-ffb8-456b-ad7d-68836088b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:22.999560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 862/1734] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:25.805799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:25.806845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:25.807388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:25.838684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] MFA Rules not processed for user `b89279618557486f9d5b358b654a906e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:25.840702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:25.853245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 873/1735] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:28.866866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:28.867703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:28.867703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:28.994853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:28.999463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:29.124543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:29.175899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 863/1736] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:29.316661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:29.319795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:29.345194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:29.513244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d540def-72ce-4c0c-83ce-dd50a61f8d9d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:29.522723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e07f6-cc6c-461a-98ab-ec287ae45920 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:29.524292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e07f6-cc6c-461a-98ab-ec287ae45920 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:29.524625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e07f6-cc6c-461a-98ab-ec287ae45920 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:29.524910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e07f6-cc6c-461a-98ab-ec287ae45920 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:29.538131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e07f6-cc6c-461a-98ab-ec287ae45920 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:29.540860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e07f6-cc6c-461a-98ab-ec287ae45920 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:29.565145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 874/1737] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:13:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:13:31.586949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:31.588177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:31.854204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cdbbb-7663-4fc4-a1a9-4d734f2efb03 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:31.874603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:31.875591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:31.875926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:31.876423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] PATH_INFO: `/v3/users/577c34ab5b6246f09c8ceb53b9296158` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:31.878124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] RBAC: Authorizing `identity:delete_user(user_id=577c34ab5b6246f09c8ceb53b9296158)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:31.882941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:32.026967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23b407-4a38-4509-ac49-664819da3737 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:32.027504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23b407-4a38-4509-ac49-664819da3737 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:32.027504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23b407-4a38-4509-ac49-664819da3737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:32.101257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577c34ab5b6246f09c8ceb53b9296158'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:32.102003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1b23b407-4a38-4509-ac49-664819da3737 None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:32.102881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577c34ab5b6246f09c8ceb53b9296158'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:32.103267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1b23b407-4a38-4509-ac49-664819da3737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:32.137012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577c34ab5b6246f09c8ceb53b9296158'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:32.138446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 875/1738] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Aug 5 05:13:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:32.138873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577c34ab5b6246f09c8ceb53b9296158'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:32.167852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577c34ab5b6246f09c8ceb53b9296158'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:32.211802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 864/1739] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:31 2022] DELETE /identity/v3/users/577c34ab5b6246f09c8ceb53b9296158 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:32.216684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:32.216977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:32.217522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:32.234331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:32.237364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:32.287231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:32.288691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:32.468954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da16e3-a0fa-44cf-8fd6-e353e161096e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:32.482583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:32.490395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:32.494537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:32.494537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:32.495339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 876/1740] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:32.496247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:32.501426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:32.522241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:32.530092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:32.553500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:32.557428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e2184-fe81-4498-8ffc-4b5c6b81a24c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:32.571162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:32.571861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:32.572146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:32.572424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:32.575731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 865/1741] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:32.577289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:32.602304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91280e1-cbd6-4a0d-b397-5385d3e17ec0 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:32.612582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:32.614239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fc2bb-0cf0-43c7-89e0-a0bb99071dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:32.614628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:32.614894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fc2bb-0cf0-43c7-89e0-a0bb99071dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:32.616725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fc2bb-0cf0-43c7-89e0-a0bb99071dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:32.621124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fc2bb-0cf0-43c7-89e0-a0bb99071dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:32.649042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fc2bb-0cf0-43c7-89e0-a0bb99071dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:32.650870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fc2bb-0cf0-43c7-89e0-a0bb99071dc3 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:32.660857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 877/1742] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:32.683960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 866/1743] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:33.299056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:33.301623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:33.339255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5c1d7-ba23-4f44-9ed5-503510f7ecbf service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:33.352927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:33.353712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:33.368158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:33.368158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] PATH_INFO: `/v3/users/0077cf68960b46dda92fe21b2465ab3c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:33.369794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0077cf68960b46dda92fe21b2465ab3c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:33.372126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:33.508377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0077cf68960b46dda92fe21b2465ab3c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:33.509528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0077cf68960b46dda92fe21b2465ab3c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:33.518975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0077cf68960b46dda92fe21b2465ab3c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:33.520475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0077cf68960b46dda92fe21b2465ab3c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:33.524126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:33.538260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0077cf68960b46dda92fe21b2465ab3c'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:33.576684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 878/1744] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:33 2022] DELETE /identity/v3/users/0077cf68960b46dda92fe21b2465ab3c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:33.580996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:33.581305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:33.581557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:33.638118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:33.644236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:33.754015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 867/1745] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:33.807736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:33.809439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:33.992420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81174d7-fb73-48b9-9dd0-e5d09a2b8e46 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:33.999067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132ad0b-96ba-4df2-b2f6-0fa1d8b4802f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:34.000135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132ad0b-96ba-4df2-b2f6-0fa1d8b4802f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:34.007233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132ad0b-96ba-4df2-b2f6-0fa1d8b4802f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:34.008262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132ad0b-96ba-4df2-b2f6-0fa1d8b4802f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:34.030411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:34.032027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:34.034651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132ad0b-96ba-4df2-b2f6-0fa1d8b4802f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:34.036434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132ad0b-96ba-4df2-b2f6-0fa1d8b4802f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:34.053415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97cb2d-f99a-4b2a-bf04-77ff51a4d5b8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:34.072277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:34.073050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:34.073375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:34.073622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 879/1746] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:34.075059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] PATH_INFO: `/v3/users/5e907263e7ad4337b7e7502c42009468` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:34.078595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e907263e7ad4337b7e7502c42009468)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:34.080664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:34.187331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e907263e7ad4337b7e7502c42009468'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:34.189394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e907263e7ad4337b7e7502c42009468'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:34.214314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e907263e7ad4337b7e7502c42009468'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:34.216376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e907263e7ad4337b7e7502c42009468'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:34.230125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e907263e7ad4337b7e7502c42009468'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:34.278603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 868/1747] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:34 2022] DELETE /identity/v3/users/5e907263e7ad4337b7e7502c42009468 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:34.283354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:34.283750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:34.284180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:34.315340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:34.317003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:34.516505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 880/1748] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:34.526299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:34.528088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:34.664628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6c301-e872-4299-b7f8-cea67a556646 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:34.685526 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:34.686290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:34.686638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:34.686920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:34.750442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:34.752615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:34.753639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:34.756324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:34.789327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 869/1749] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:34.803560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde107e-f1e5-4c0f-92ca-923a113f0e55 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:34.810745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:34.812024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:34.812514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:34.813329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] PATH_INFO: `/v3/projects/368da15828c243f896200dde57aa68f5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:34.816464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=368da15828c243f896200dde57aa68f5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:34.818546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: WARNING py.warnings [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 05:13:34.818546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: warnings.warn(msg) Aug 05 05:13:34.818546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: Aug 05 05:13:34.821384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:35.061848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368da15828c243f896200dde57aa68f5', 'request_id': 'req-1a22c390-cf1d-4d1c-8dd4-f27976846f30'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.080395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368da15828c243f896200dde57aa68f5', 'request_id': 'req-1a22c390-cf1d-4d1c-8dd4-f27976846f30'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.134486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368da15828c243f896200dde57aa68f5', 'request_id': 'req-1a22c390-cf1d-4d1c-8dd4-f27976846f30'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.181424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] The token cache is being invalidate because project 368da15828c243f896200dde57aa68f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:35.181847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.183915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.188895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 881/1750] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:34 2022] DELETE /identity/v3/projects/368da15828c243f896200dde57aa68f5 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:35.192996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:35.211049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aba09-2e05-4ed7-b1ef-ddd0fd59367c service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:35.213030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:35.386122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:35.387068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:35.387545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:35.388035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] PATH_INFO: `/v3/users/76c149bc0cbe425287397ac5e24534ff` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:35.389777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c149bc0cbe425287397ac5e24534ff)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:35.392395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:35.484476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c149bc0cbe425287397ac5e24534ff'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.485558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c149bc0cbe425287397ac5e24534ff'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.519081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c149bc0cbe425287397ac5e24534ff'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.522375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c149bc0cbe425287397ac5e24534ff'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.537622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c149bc0cbe425287397ac5e24534ff'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:35.586051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 870/1751] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:35 2022] DELETE /identity/v3/users/76c149bc0cbe425287397ac5e24534ff => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:36.173433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:36.174996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:36.188314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:36.198063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22c390-cf1d-4d1c-8dd4-f27976846f30 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:36.327371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:36.331221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:36.332672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:36.333629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] PATH_INFO: `/v3/projects/cb67419a21c94f158aac708300604332` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:36.338886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb67419a21c94f158aac708300604332)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:36.341300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:36.498759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb67419a21c94f158aac708300604332', 'request_id': 'req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.527566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb67419a21c94f158aac708300604332', 'request_id': 'req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.545843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb67419a21c94f158aac708300604332', 'request_id': 'req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.583213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] The token cache is being invalidate because project cb67419a21c94f158aac708300604332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:36.583493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.584224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f3084e3b-e3a1-4afb-b9f5-15780d5e7fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.586438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 882/1752] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:36 2022] DELETE /identity/v3/projects/cb67419a21c94f158aac708300604332 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:36.590839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:36.600371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:36.646129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d68d8-8884-4112-bbf9-e2fcb3851aa2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:36.674494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec4559-1c16-4a6d-b31f-bbc063967598 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:36.674989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec4559-1c16-4a6d-b31f-bbc063967598 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:36.675329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec4559-1c16-4a6d-b31f-bbc063967598 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:36.678590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 883/1753] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:13:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:36.783978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:36.784328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:36.784612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:36.784892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] PATH_INFO: `/v3/users/ee714c90af564a7881a7d5e27ca0d4f0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:36.786767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee714c90af564a7881a7d5e27ca0d4f0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:36.789789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:36.827165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:36.859397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:36.860285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:36.862775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:36.921183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee714c90af564a7881a7d5e27ca0d4f0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.922834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee714c90af564a7881a7d5e27ca0d4f0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.925054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:36.927836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:36.942933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee714c90af564a7881a7d5e27ca0d4f0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.944929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee714c90af564a7881a7d5e27ca0d4f0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:36.962006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee714c90af564a7881a7d5e27ca0d4f0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:37.003843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 871/1754] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:36 2022] DELETE /identity/v3/users/ee714c90af564a7881a7d5e27ca0d4f0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:37.054538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 884/1755] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:13:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:37.506764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:37.508950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:37.700932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f3c4-e117-4151-9c23-bcab0e03bb22 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:37.711041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:37.713277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:37.713603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:37.718777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] PATH_INFO: `/v3/projects/15b2bc917952432b8c538f07bbb6729d` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:37.722503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b2bc917952432b8c538f07bbb6729d)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:37.726074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:37.831007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b2bc917952432b8c538f07bbb6729d', 'request_id': 'req-2fad2184-ff72-4d78-afa3-841da67210a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:37.845260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b2bc917952432b8c538f07bbb6729d', 'request_id': 'req-2fad2184-ff72-4d78-afa3-841da67210a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:37.865768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b2bc917952432b8c538f07bbb6729d', 'request_id': 'req-2fad2184-ff72-4d78-afa3-841da67210a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:37.897130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] The token cache is being invalidate because project 15b2bc917952432b8c538f07bbb6729d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:37.897357 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:37.898031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:37.901161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 872/1756] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:37 2022] DELETE /identity/v3/projects/15b2bc917952432b8c538f07bbb6729d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:37.905092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:37.927696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7d84c-5368-4a6d-a3df-102dad15fd5c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:37.974384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:38.069915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:38.070595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:38.070990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:38.071273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] PATH_INFO: `/v3/users/824b8a40bccd4749826c7e535da039b3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:38.075603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=824b8a40bccd4749826c7e535da039b3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:38.078331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:38.174025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:38.175522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:38.182598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824b8a40bccd4749826c7e535da039b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.184880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824b8a40bccd4749826c7e535da039b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.203858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad2184-ff72-4d78-afa3-841da67210a0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:38.204061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824b8a40bccd4749826c7e535da039b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.207431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824b8a40bccd4749826c7e535da039b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.212760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:38.213894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:38.214214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:38.214732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] PATH_INFO: `/v3/projects/ffd9e9e15b914368b959a786815e2ea6` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:38.218069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd9e9e15b914368b959a786815e2ea6)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:38.223119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:38.228006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250a8f3f-ecdc-4728-8439-61d3cea53983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824b8a40bccd4749826c7e535da039b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.288354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 885/1757] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:37 2022] DELETE /identity/v3/users/824b8a40bccd4749826c7e535da039b3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:38.344463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:38.421260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd9e9e15b914368b959a786815e2ea6', 'request_id': 'req-1e8729f1-413d-489c-bc28-720e00cc02be'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.450698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd9e9e15b914368b959a786815e2ea6', 'request_id': 'req-1e8729f1-413d-489c-bc28-720e00cc02be'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.462200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd9e9e15b914368b959a786815e2ea6', 'request_id': 'req-1e8729f1-413d-489c-bc28-720e00cc02be'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.502840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] The token cache is being invalidate because project ffd9e9e15b914368b959a786815e2ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:38.503034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.504800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:38.513225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 873/1758] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:38 2022] DELETE /identity/v3/projects/ffd9e9e15b914368b959a786815e2ea6 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:38.516053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:38.516873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:38.517175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:38.541163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:38.548793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:38.622355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:38.626380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:38.768231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8729f1-413d-489c-bc28-720e00cc02be admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:38.783650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:38.785860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:38.786247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:38.786556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:38.798758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 886/1759] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:38.798758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:38.798758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:38.806137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:38.833539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01899-9cf3-4c68-b0fc-c577e9c73c2a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:38.849191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:38.851141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecaf39-f815-4f82-aadc-e044c20936f4 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:38.870019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 874/1760] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:38.877070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:38.877592 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:38.878092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:38.881726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:38.882505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:38.882931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:38.883289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] PATH_INFO: `/v3/projects/0f0259cdb1e94a08b91c41aadd43d7c2` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:38.887824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0259cdb1e94a08b91c41aadd43d7c2)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:38.887824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:38.941044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:38.942910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:38.955313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 875/1761] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:38.960639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:38.986874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0da971-1c33-465b-afd9-2c3fbc0b324b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.007014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:39.008108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.008772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.009242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.010168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:39.010693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:39.013034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2876b-12f8-4509-951d-9474370f86ba admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:39.021998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:39.049114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0259cdb1e94a08b91c41aadd43d7c2', 'request_id': 'req-d07dd248-5ecd-4169-a35e-e3016524c49d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:39.049956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 876/1762] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:39.053226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185102d-9a86-4b4f-adff-7f891feea30b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.055983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185102d-9a86-4b4f-adff-7f891feea30b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.056489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185102d-9a86-4b4f-adff-7f891feea30b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.062899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 877/1763] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:13:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:39.063525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0259cdb1e94a08b91c41aadd43d7c2', 'request_id': 'req-d07dd248-5ecd-4169-a35e-e3016524c49d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:39.064226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185102d-9a86-4b4f-adff-7f891feea30b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:39.081736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0259cdb1e94a08b91c41aadd43d7c2', 'request_id': 'req-d07dd248-5ecd-4169-a35e-e3016524c49d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:39.111325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] The token cache is being invalidate because project 0f0259cdb1e94a08b91c41aadd43d7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:39.112903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:39.113783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d07dd248-5ecd-4169-a35e-e3016524c49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:39.118764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 887/1764] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:38 2022] DELETE /identity/v3/projects/0f0259cdb1e94a08b91c41aadd43d7c2 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:39.124161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4906e-e335-4126-8e0b-8563b3c58a0a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.124473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4906e-e335-4126-8e0b-8563b3c58a0a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.124842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4906e-e335-4126-8e0b-8563b3c58a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.169224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-39a4906e-e335-4126-8e0b-8563b3c58a0a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:39.170570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-39a4906e-e335-4126-8e0b-8563b3c58a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:39.236490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185102d-9a86-4b4f-adff-7f891feea30b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.245949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4caf37-8f1b-43a2-97c3-a4ce8d3cea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:39.246602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4caf37-8f1b-43a2-97c3-a4ce8d3cea5d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.246882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4caf37-8f1b-43a2-97c3-a4ce8d3cea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.247148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4caf37-8f1b-43a2-97c3-a4ce8d3cea5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.247846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4caf37-8f1b-43a2-97c3-a4ce8d3cea5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:39.249772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4caf37-8f1b-43a2-97c3-a4ce8d3cea5d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:39.263819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 888/1765] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:39.265424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.265716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.265995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.295318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:39.296697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:39.334806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 878/1766] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/projects => generated 369 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:39.336762 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.337204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.349575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.364772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:39.426381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:39.430831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:39.510317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 889/1767] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:39.511803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:39.521325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:39.544835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e3acd-cb22-4c7a-a176-e420b071096e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.609188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 879/1768] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:39.610575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:39.622744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08b8b-1ff1-4bdc-bef9-a62286b6ee9f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.629738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:39.630466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.630836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.631114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.631933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:39.632243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:39.634031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:39.653697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 880/1769] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:39.654686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:39.666140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca5df-4217-4595-aa3a-919dd705c8f7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.676214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:39.676999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.677330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.677600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.678396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:39.680651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:39.684056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:39.684913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.685262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.685538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.686356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:39.693446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:39.703571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:39.733943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 890/1770] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:39.739403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:39.758397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 881/1771] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:39.759527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:39.776579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611e55-d3f2-429b-b88f-0588224ec8e0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.948861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f259c-1f70-4c8c-b9de-ad3008d88a28 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:39.956358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:39.956665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:39.957605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:39.957605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:39.958671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:39.963846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.015790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 882/1772] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:39 2022] POST /identity/v3/users => generated 284 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:40.016992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.026423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.027116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.027471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.027691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.028687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.034150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.042841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841a24a-cec1-480f-ac8e-d2b797b9d57f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.057982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.061014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.061367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.061611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.062482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.064505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.065605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 891/1773] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:40.071536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.101127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2c4c-7349-425c-83ff-4fdaa910bc33 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.102991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 883/1774] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:40.105036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.124102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb4465-2615-4f56-bfb8-02f58f91d626 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.131814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.132750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.133224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.133467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.134304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.134434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.134701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.134975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.135687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.136794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.137319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.138813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.151178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 884/1775] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:40.157298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.164440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 892/1776] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:40.172342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.180984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985ef5-13db-4d21-b80a-dabc1010c816 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.191430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91b4e3-4b4a-4904-b310-c2fb2ce3cfe0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.205239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.206687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.208263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.208690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] PATH_INFO: `/v3/projects/af374482606e45cfa8c0de626d71dc90/users/db2e1266dc6f4ea18486c1b26dcaf9ea/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.220509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7bf4b-fdb2-4845-9be2-acb831fcb5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.224441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7bf4b-fdb2-4845-9be2-acb831fcb5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.224831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7bf4b-fdb2-4845-9be2-acb831fcb5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.225157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7bf4b-fdb2-4845-9be2-acb831fcb5ea admin admin] PATH_INFO: `/v3/projects/75142bcd0b7947b5a0f616d8604f330f/users/e66b8340c32747428306fd0ed90d5b69/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.269147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af374482606e45cfa8c0de626d71dc90, user_id=db2e1266dc6f4ea18486c1b26dcaf9ea, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.278981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.280446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7bf4b-fdb2-4845-9be2-acb831fcb5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=75142bcd0b7947b5a0f616d8604f330f, user_id=e66b8340c32747428306fd0ed90d5b69, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.290967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7bf4b-fdb2-4845-9be2-acb831fcb5ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.312753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 885/1777] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:40 2022] PUT /identity/v3/projects/af374482606e45cfa8c0de626d71dc90/users/db2e1266dc6f4ea18486c1b26dcaf9ea/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:40.314183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.316515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:40.326459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 893/1778] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:40 2022] PUT /identity/v3/projects/75142bcd0b7947b5a0f616d8604f330f/users/e66b8340c32747428306fd0ed90d5b69/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:40.328287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbde479-61f5-4d51-86d3-358d2fe6413c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.328744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbde479-61f5-4d51-86d3-358d2fe6413c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.329187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbde479-61f5-4d51-86d3-358d2fe6413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.375349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:40.457667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3dbde479-61f5-4d51-86d3-358d2fe6413c None None] MFA Rules not processed for user `db2e1266dc6f4ea18486c1b26dcaf9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:40.467697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3dbde479-61f5-4d51-86d3-358d2fe6413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:40.575410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205e1c-2f7d-4917-a0ba-86462027dd47 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.653750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 894/1779] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:40.653750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.654517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.654860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.734626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.734888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] MFA Rules not processed for user `e66b8340c32747428306fd0ed90d5b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:40.735792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.736265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.736723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.738889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:40.749911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.752662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.767528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 886/1780] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:40.768989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.838606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:40.937017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 895/1781] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:40.938254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:40.949456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004fd69-03b1-476a-8138-42f7005a207b service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.966297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dfdec-a014-4b63-a21e-e9e2d2034ae9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:40.968663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.970174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.972223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.974135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.975291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.983661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:40.984601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:40.986102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:40.987219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:40.987674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:40.990305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:40.993435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.059486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 887/1782] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:40 2022] POST /identity/v3/projects => generated 369 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:41.060859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.078674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 896/1783] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:40 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:41.079918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.121287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:41.207241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf72bd-7747-4190-80b1-2db5503f63ba admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.217433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.219348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.220417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.220417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.223393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.224126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7fbc8-0063-4c27-af26-e777b98fa935 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.227474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.238486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.239247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.239539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.239973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.240666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.247111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.290166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 888/1784] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:41 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:41.294890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.319946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1d373-bcf1-4504-b8ba-5e22d5b5e52c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.324655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 897/1785] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:41 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:41.331343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.334485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.335144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.335427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.335830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.336553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.338335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.344070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5252d6-fa64-43cc-ac52-c72b5dfa87f4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.357460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.358831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.359456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.369902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.374534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.376339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 889/1786] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:41.377352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.379966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.404960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb946ac-98d3-458d-b402-13b9b0dec72c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.420437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.420985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 898/1787] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:41.421173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.421482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.421813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] PATH_INFO: `/v3/projects/c56843fa38b34a4ba9e710cb962e3064/users/6c9e35df8e7f46a395a44a67c302c969/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.423940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.448854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd041b4-d8e5-438e-99ed-9ce674d95693 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.459410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.461111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.461783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.462324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] PATH_INFO: `/v3/projects/48d75ee160a34744a4d8e391f62d4d70/users/ed7cf847cdc04a4c891b0e62c20b275f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.464408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56843fa38b34a4ba9e710cb962e3064, user_id=6c9e35df8e7f46a395a44a67c302c969, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.467634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.483338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d75ee160a34744a4d8e391f62d4d70, user_id=ed7cf847cdc04a4c891b0e62c20b275f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.486968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.504224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 890/1788] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:41 2022] PUT /identity/v3/projects/c56843fa38b34a4ba9e710cb962e3064/users/6c9e35df8e7f46a395a44a67c302c969/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:41.509117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.529864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 899/1789] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:41 2022] PUT /identity/v3/projects/48d75ee160a34744a4d8e391f62d4d70/users/ed7cf847cdc04a4c891b0e62c20b275f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:41.531956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.652869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f06b2-09d7-4d89-bf3f-475cd5a74122 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.653172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fab90-3947-4bae-b436-72da563765cc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.659432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.660111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.660325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.660465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.660858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.661672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.661797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.662158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.662741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.663285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.663708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.666360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.670871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:41.679686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 891/1790] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:41.680765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 900/1791] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:41.684938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.685450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.698364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71e9e2-67a7-43a6-8018-d9598fd86b1d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.698541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea64938-1311-4d36-bee5-af5590ee5ce9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:41.704392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09217d7b-0f43-4898-a0c6-d5a11d4205ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.704793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09217d7b-0f43-4898-a0c6-d5a11d4205ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.705159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09217d7b-0f43-4898-a0c6-d5a11d4205ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.705494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09217d7b-0f43-4898-a0c6-d5a11d4205ac admin admin] PATH_INFO: `/v3/projects/c56843fa38b34a4ba9e710cb962e3064/users/6c9e35df8e7f46a395a44a67c302c969/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.705611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a6ea0-6436-465f-b39a-8b089e693338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:41.706893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a6ea0-6436-465f-b39a-8b089e693338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.707455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a6ea0-6436-465f-b39a-8b089e693338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.707658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09217d7b-0f43-4898-a0c6-d5a11d4205ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56843fa38b34a4ba9e710cb962e3064, user_id=6c9e35df8e7f46a395a44a67c302c969, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.707939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a6ea0-6436-465f-b39a-8b089e693338 admin admin] PATH_INFO: `/v3/projects/48d75ee160a34744a4d8e391f62d4d70/users/ed7cf847cdc04a4c891b0e62c20b275f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.710614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09217d7b-0f43-4898-a0c6-d5a11d4205ac admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.710726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a6ea0-6436-465f-b39a-8b089e693338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d75ee160a34744a4d8e391f62d4d70, user_id=ed7cf847cdc04a4c891b0e62c20b275f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:41.714046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a6ea0-6436-465f-b39a-8b089e693338 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:41.726851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 892/1792] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:41 2022] PUT /identity/v3/projects/c56843fa38b34a4ba9e710cb962e3064/users/6c9e35df8e7f46a395a44a67c302c969/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:41.728613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 901/1793] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:41 2022] PUT /identity/v3/projects/48d75ee160a34744a4d8e391f62d4d70/users/ed7cf847cdc04a4c891b0e62c20b275f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:41.737894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.738356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.738928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.739950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:41.740244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:41.740519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:41.774254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] MFA Rules not processed for user `ed7cf847cdc04a4c891b0e62c20b275f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:41.776021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:41.781353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] MFA Rules not processed for user `6c9e35df8e7f46a395a44a67c302c969`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:41.783814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:41.877808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:41.908128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 902/1794] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:41.908128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.908128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:41.913141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 893/1795] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:41.925706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:41.927917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:41.999298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18ebd6-73b8-4ed7-9db2-f289f3bb9180 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.004651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.005295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.005693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.005972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.006866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d9f8b-b998-456d-947a-1c5877917801 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.016081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.017205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.017521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.017884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.018160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.018436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] PATH_INFO: `/v3/projects/350f89ff304a457281b466344d58d27a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.019657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=350f89ff304a457281b466344d58d27a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.021793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.029233 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 894/1796] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:42.030906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.032728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:42.046061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95844cd8-3927-422c-b0cd-6574b8bd5b96 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.050942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.052451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.052890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.053088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.064020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.066187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.078684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 895/1797] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:42.100337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350f89ff304a457281b466344d58d27a', 'request_id': 'req-7dac8ca0-c988-469c-8fce-88b83574d434'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:42.106669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350f89ff304a457281b466344d58d27a', 'request_id': 'req-7dac8ca0-c988-469c-8fce-88b83574d434'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:42.114348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350f89ff304a457281b466344d58d27a', 'request_id': 'req-7dac8ca0-c988-469c-8fce-88b83574d434'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:42.135535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] The token cache is being invalidate because project 350f89ff304a457281b466344d58d27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:42.135879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:42.137548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7dac8ca0-c988-469c-8fce-88b83574d434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:42.140634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 903/1798] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:41 2022] DELETE /identity/v3/projects/350f89ff304a457281b466344d58d27a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:42.141432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:42.144345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.146145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee070e8f-2762-4d90-9e05-96ac38287a6a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.146438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee070e8f-2762-4d90-9e05-96ac38287a6a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.146726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee070e8f-2762-4d90-9e05-96ac38287a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.156764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8be9-22e4-4e73-807a-5c39fad3f5f7 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.173560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ee070e8f-2762-4d90-9e05-96ac38287a6a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:42.175415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ee070e8f-2762-4d90-9e05-96ac38287a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:42.275364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.276052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.276340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.276623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.281170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:42.286225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 904/1799] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:42.296101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf771069-f379-41fe-9250-92392489545a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.296626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf771069-f379-41fe-9250-92392489545a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.297099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf771069-f379-41fe-9250-92392489545a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.325752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bf771069-f379-41fe-9250-92392489545a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:42.327411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf771069-f379-41fe-9250-92392489545a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:42.347216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 905/1800] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:42.350430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf771069-f379-41fe-9250-92392489545a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.363634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf771069-f379-41fe-9250-92392489545a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.369985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.370638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.370918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.371188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.372072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:42.372538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.372684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.374390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.377208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.395295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 906/1801] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:42.396138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 896/1802] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:42.399416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.413666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3765-c4bd-491e-8360-14afa02a48a5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.423610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.424812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.425095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.425421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.426263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.428408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.488195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 907/1803] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:42.492939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.508824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a2bb1-46a5-4ad4-b8f0-0b5496f9a8da service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.539024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:42.573298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.574072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.574386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.574718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.575579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.577925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.603831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 897/1804] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:42 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:42.605871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.616887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5630f2-905f-46b8-84b7-b4aacb404b95 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.621915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.622598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.622923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.623139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.623992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.625676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.637331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 908/1805] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:42.640898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.651523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b20a1-eaea-43a9-a7fd-d4fc9b947c9d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.657339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e2a5b-a646-4a08-a627-0f49aced70b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.657974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2a5b-a646-4a08-a627-0f49aced70b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.658247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2a5b-a646-4a08-a627-0f49aced70b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.658556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2a5b-a646-4a08-a627-0f49aced70b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.659259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e2a5b-a646-4a08-a627-0f49aced70b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.660975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e2a5b-a646-4a08-a627-0f49aced70b3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.672670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 898/1806] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:42.677479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:42.687911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc3af9-a891-4306-ac1d-93a6797444b3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:42.703722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:42.704469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.704772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.705026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] PATH_INFO: `/v3/projects/f80d559e1b6443f88b23960e2c2c7c89/users/8dc0ff1fbffb46b79abaf2afd9f54f63/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.725219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80d559e1b6443f88b23960e2c2c7c89, user_id=8dc0ff1fbffb46b79abaf2afd9f54f63, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:42.727949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:42.768007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 909/1807] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:42 2022] PUT /identity/v3/projects/f80d559e1b6443f88b23960e2c2c7c89/users/8dc0ff1fbffb46b79abaf2afd9f54f63/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:42.781053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:42.781350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:42.781630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:42.830079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] MFA Rules not processed for user `8dc0ff1fbffb46b79abaf2afd9f54f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:42.833572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:42.956877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 899/1808] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:42.961356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.066422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78e188-d962-4562-a67b-06e462fe1159 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.071195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.071849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.072135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.072412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.076596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.078600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.123870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 910/1809] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:43.127733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.138728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84628a48-8c24-40f0-8355-80f8aac1cad8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.213933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.214676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.215072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.215725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.216205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.218197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.248547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 900/1810] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:43 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:43.249744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.268374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7e61f-6406-442b-8dad-582ecebbadf5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.272956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.273585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.274022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.274298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.275076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.276773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.279221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:43.292098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 911/1811] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:43.296298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.309569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdca0bc-b526-41ee-b3d5-2831f2a91ea0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.314343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.315029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.315310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.315583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] PATH_INFO: `/v3/projects/d62627d9f8464a40ad53565716dd440b/users/cab1aa2cf6394e7bbe4a06c850e4280f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.332538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62627d9f8464a40ad53565716dd440b, user_id=cab1aa2cf6394e7bbe4a06c850e4280f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.337299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.354739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 901/1812] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:43 2022] PUT /identity/v3/projects/d62627d9f8464a40ad53565716dd440b/users/cab1aa2cf6394e7bbe4a06c850e4280f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:43.358488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.370525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea707eb-113f-40bd-85b1-1dc782f71f06 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.449795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318638b-ed88-49e9-b91e-23f1f5181de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.450912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318638b-ed88-49e9-b91e-23f1f5181de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.451291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318638b-ed88-49e9-b91e-23f1f5181de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.451572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318638b-ed88-49e9-b91e-23f1f5181de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.452423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318638b-ed88-49e9-b91e-23f1f5181de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.454047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318638b-ed88-49e9-b91e-23f1f5181de8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.464825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 912/1813] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:43.469956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.474407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:43.498476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b0b42-c416-49b4-a479-ba9b76853f84 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.505910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.506652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.506940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.507219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] PATH_INFO: `/v3/projects/d62627d9f8464a40ad53565716dd440b/users/cab1aa2cf6394e7bbe4a06c850e4280f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.509398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62627d9f8464a40ad53565716dd440b, user_id=cab1aa2cf6394e7bbe4a06c850e4280f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.512139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.528199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 902/1814] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:43 2022] PUT /identity/v3/projects/d62627d9f8464a40ad53565716dd440b/users/cab1aa2cf6394e7bbe4a06c850e4280f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:43.538674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.538962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.539238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.574432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] MFA Rules not processed for user `cab1aa2cf6394e7bbe4a06c850e4280f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:43.576108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:43.677899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 913/1815] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:43.693464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.697018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:43.770310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a30440-a754-4eb9-af50-021ee08258a6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.777837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.779293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.781455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.781455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.795955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.797699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.807343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 903/1816] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:43.837806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.893679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:43.895375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:43.910954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d64bec-6bdf-467b-bf68-c602b0d2cc2f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.916769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.920642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.921072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] PATH_INFO: `/v3/users/8dc0ff1fbffb46b79abaf2afd9f54f63` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.922921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dc0ff1fbffb46b79abaf2afd9f54f63)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.925232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.950758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:43.974407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7bb0c-01e7-4e1b-8686-c6bf32fce37f service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:43.984006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:43.984727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:43.985058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:43.985340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] PATH_INFO: `/v3/users/e66b8340c32747428306fd0ed90d5b69` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:43.985504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dc0ff1fbffb46b79abaf2afd9f54f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:43.986152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dc0ff1fbffb46b79abaf2afd9f54f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:43.986554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e66b8340c32747428306fd0ed90d5b69)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:43.988789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:43.994494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dc0ff1fbffb46b79abaf2afd9f54f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:43.995486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dc0ff1fbffb46b79abaf2afd9f54f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.007104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9a1825ac-2eac-46ae-bc2d-2ad04d650093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dc0ff1fbffb46b79abaf2afd9f54f63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.030569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 914/1817] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:43 2022] DELETE /identity/v3/users/8dc0ff1fbffb46b79abaf2afd9f54f63 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:44.032170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:44.032467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:44.032741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:44.062598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:44.063982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:44.071564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e66b8340c32747428306fd0ed90d5b69'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.072311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e66b8340c32747428306fd0ed90d5b69'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.084593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e66b8340c32747428306fd0ed90d5b69'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.085378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e66b8340c32747428306fd0ed90d5b69'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.098816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ee259780-146c-461e-a9d5-ebf454e1c726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e66b8340c32747428306fd0ed90d5b69'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.137686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 904/1818] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:43 2022] DELETE /identity/v3/users/e66b8340c32747428306fd0ed90d5b69 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:44.139632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:44.139955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:44.140239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:44.157546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 915/1819] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:44.169933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:44.171285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:44.172042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:44.181603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:44.197279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:44.243188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274aa8a2-d658-4ace-825a-823542cb7fdd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:44.300991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:44.302253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:44.303081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:44.303680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:44.312983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 905/1820] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:44.322982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:44.324798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:44.326529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:44.331688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:44.343299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 916/1821] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:13:44.356052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15afd7-7021-404d-aa79-66d027008029 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:44.360483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:44.361261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:44.361704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:44.362052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:44.374117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:44.375626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:44.386186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 906/1822] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:44.881177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:44.882643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:44.887866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:44.889373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:44.902358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc999c4-0ce2-41f5-83ac-c32796860bf7 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:44.908033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:44.908687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:44.908968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:44.909289 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] PATH_INFO: `/v3/projects/f80d559e1b6443f88b23960e2c2c7c89` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:44.910538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80d559e1b6443f88b23960e2c2c7c89)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:44.912756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:44.926587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05cdec-41d3-46f6-b35a-9f76fc26f446 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:44.933206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:44.934111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:44.934489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:44.934766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] PATH_INFO: `/v3/projects/75142bcd0b7947b5a0f616d8604f330f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:44.935968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] RBAC: Authorizing `identity:delete_project(project_id=75142bcd0b7947b5a0f616d8604f330f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:44.937880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:44.966032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80d559e1b6443f88b23960e2c2c7c89', 'request_id': 'req-e3abebd5-0f39-4786-b1a6-017efc1ca490'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.972845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80d559e1b6443f88b23960e2c2c7c89', 'request_id': 'req-e3abebd5-0f39-4786-b1a6-017efc1ca490'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.979417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80d559e1b6443f88b23960e2c2c7c89', 'request_id': 'req-e3abebd5-0f39-4786-b1a6-017efc1ca490'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.986348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75142bcd0b7947b5a0f616d8604f330f', 'request_id': 'req-35cbce2a-92f5-46eb-ae11-4eb4350f184d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.992666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75142bcd0b7947b5a0f616d8604f330f', 'request_id': 'req-35cbce2a-92f5-46eb-ae11-4eb4350f184d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.993482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] The token cache is being invalidate because project f80d559e1b6443f88b23960e2c2c7c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:44.994077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.995395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:44.999895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 917/1823] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:44 2022] DELETE /identity/v3/projects/f80d559e1b6443f88b23960e2c2c7c89 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:45.000025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75142bcd0b7947b5a0f616d8604f330f', 'request_id': 'req-35cbce2a-92f5-46eb-ae11-4eb4350f184d'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.002324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:45.004411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:45.020108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] The token cache is being invalidate because project 75142bcd0b7947b5a0f616d8604f330f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:45.020435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.021227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.022786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 907/1824] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:44 2022] DELETE /identity/v3/projects/75142bcd0b7947b5a0f616d8604f330f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:45.026784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:45.028225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:45.070520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abebd5-0f39-4786-b1a6-017efc1ca490 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:45.092001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbce2a-92f5-46eb-ae11-4eb4350f184d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:45.096752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:45.097469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:45.097853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:45.098196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] PATH_INFO: `/v3/users/ed7cf847cdc04a4c891b0e62c20b275f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:45.099445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7cf847cdc04a4c891b0e62c20b275f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:45.101413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:45.124226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:45.128777 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:45.129510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:45.129807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:45.130109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] PATH_INFO: `/v3/users/cab1aa2cf6394e7bbe4a06c850e4280f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:45.131492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab1aa2cf6394e7bbe4a06c850e4280f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:45.133477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:45.152920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7cf847cdc04a4c891b0e62c20b275f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.153707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7cf847cdc04a4c891b0e62c20b275f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.161449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7cf847cdc04a4c891b0e62c20b275f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.162289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7cf847cdc04a4c891b0e62c20b275f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.170704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:45.171229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7cf847cdc04a4c891b0e62c20b275f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.186805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab1aa2cf6394e7bbe4a06c850e4280f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.187647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab1aa2cf6394e7bbe4a06c850e4280f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.193313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 908/1825] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:45 2022] DELETE /identity/v3/users/ed7cf847cdc04a4c891b0e62c20b275f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:45.195897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab1aa2cf6394e7bbe4a06c850e4280f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.197384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab1aa2cf6394e7bbe4a06c850e4280f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.205854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab1aa2cf6394e7bbe4a06c850e4280f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:45.223324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 918/1826] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:45 2022] DELETE /identity/v3/users/cab1aa2cf6394e7bbe4a06c850e4280f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:46.001615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.002663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.080556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.082067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.146406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f700e8-475b-4200-9062-3fce339e4a3c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.217391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95ff65-746b-4022-9ffe-ee7bce625c33 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.230904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.231580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.231887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.232162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] PATH_INFO: `/v3/projects/d62627d9f8464a40ad53565716dd440b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.233373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62627d9f8464a40ad53565716dd440b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.235607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.290718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.291291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.291587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.291939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] PATH_INFO: `/v3/projects/48d75ee160a34744a4d8e391f62d4d70` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.299983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d75ee160a34744a4d8e391f62d4d70)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.304443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.329003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62627d9f8464a40ad53565716dd440b', 'request_id': 'req-5a9ed23c-387b-4054-9430-ef9e46f8d33d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.348021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62627d9f8464a40ad53565716dd440b', 'request_id': 'req-5a9ed23c-387b-4054-9430-ef9e46f8d33d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.357023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62627d9f8464a40ad53565716dd440b', 'request_id': 'req-5a9ed23c-387b-4054-9430-ef9e46f8d33d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.378367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] The token cache is being invalidate because project d62627d9f8464a40ad53565716dd440b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:46.378994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.380360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.382843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 919/1827] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:46 2022] DELETE /identity/v3/projects/d62627d9f8464a40ad53565716dd440b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:46.384611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.386284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.420694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d75ee160a34744a4d8e391f62d4d70', 'request_id': 'req-219e9dab-398c-4a07-ade9-03098194c7c3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.424561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:46.449036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:46.450421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d75ee160a34744a4d8e391f62d4d70', 'request_id': 'req-219e9dab-398c-4a07-ade9-03098194c7c3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.468066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d75ee160a34744a4d8e391f62d4d70', 'request_id': 'req-219e9dab-398c-4a07-ade9-03098194c7c3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.488154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed23c-387b-4054-9430-ef9e46f8d33d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.514326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] The token cache is being invalidate because project 48d75ee160a34744a4d8e391f62d4d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:46.515680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.517171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-219e9dab-398c-4a07-ade9-03098194c7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.519737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 909/1828] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:45 2022] DELETE /identity/v3/projects/48d75ee160a34744a4d8e391f62d4d70 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:46.521267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ae34-f27a-444c-bcdc-1d2550c2f1af None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.521701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ae34-f27a-444c-bcdc-1d2550c2f1af None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.522106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ae34-f27a-444c-bcdc-1d2550c2f1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.549354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8b25ae34-f27a-444c-bcdc-1d2550c2f1af None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:46.551682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8b25ae34-f27a-444c-bcdc-1d2550c2f1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.556487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.557160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.557470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.557743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] PATH_INFO: `/v3/users/db2e1266dc6f4ea18486c1b26dcaf9ea` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.558944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2e1266dc6f4ea18486c1b26dcaf9ea)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.560931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.561151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 910/1829] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:46.562838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aba14d-969d-40cd-b394-dd75b5547856 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.563262 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aba14d-969d-40cd-b394-dd75b5547856 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.564156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aba14d-969d-40cd-b394-dd75b5547856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.588649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-15aba14d-969d-40cd-b394-dd75b5547856 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:46.590184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-15aba14d-969d-40cd-b394-dd75b5547856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.605499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 911/1830] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:46.605499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db21c44-0007-4f87-a262-1cee8831fff9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.605499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db21c44-0007-4f87-a262-1cee8831fff9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.605499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db21c44-0007-4f87-a262-1cee8831fff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.621072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2e1266dc6f4ea18486c1b26dcaf9ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.621409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6db21c44-0007-4f87-a262-1cee8831fff9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:46.621759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2e1266dc6f4ea18486c1b26dcaf9ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.622610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6db21c44-0007-4f87-a262-1cee8831fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.630888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 912/1831] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:46.632227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5de2c-d17b-464d-9004-7fc83cf22e37 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.632514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5de2c-d17b-464d-9004-7fc83cf22e37 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.632790 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5de2c-d17b-464d-9004-7fc83cf22e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.635065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2e1266dc6f4ea18486c1b26dcaf9ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.636268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2e1266dc6f4ea18486c1b26dcaf9ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.646437 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2e1266dc6f4ea18486c1b26dcaf9ea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:46.655360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0ad5de2c-d17b-464d-9004-7fc83cf22e37 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:46.656613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0ad5de2c-d17b-464d-9004-7fc83cf22e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.664750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 920/1832] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:46 2022] DELETE /identity/v3/users/db2e1266dc6f4ea18486c1b26dcaf9ea => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:46.668836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.668957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 913/1833] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:46.669347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.669775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.670108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.691093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:46.692424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.735953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21773147-29c8-4304-955f-9e2aec2ec580 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.747649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.748342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.748624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.748895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.749640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:46.749947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.751573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.757378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 914/1834] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:46.762578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.762788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:46.768204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6954a-f27e-4721-b33c-10559114edfb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.770054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 921/1835] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:46.771116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.774685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:46.777151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.777716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.778028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.778276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.778976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:46.779324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.780915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.794562 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 915/1836] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:46.795525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.805373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d7d2-b894-4aad-b0a2-d8fe0f271e29 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.810094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.810725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.811008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.811281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.811993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.813927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.841179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c75b-9c97-4620-962a-7e66900a92ef admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.843049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 916/1837] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:46.843953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.893197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.893828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.894110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.894389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.913007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ee76f-74f2-4a3a-803a-e8d50669623d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:46.918265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:46.918899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:46.919184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:46.919461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:46.920169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.922083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.952864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 917/1838] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:46.953864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:46.959215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:46.969013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:46.972364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:46.984539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 922/1839] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:46.985592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.003460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a70f-9b3c-4231-b8c4-472f438135e3 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.088727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.089611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.090086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.090532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.091660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.094020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.104517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181af92-eb15-4af2-98a2-08f7bd5594bc admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.105827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:47.114767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.115350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.115867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.116155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.116997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.119094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.123776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 923/1840] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:47.128288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.140711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299c1ef-cbbf-418e-be8e-837b8d6f527d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.145740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.146420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.146709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.146990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.147808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 918/1841] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:46 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:47.147957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.149394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.151832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.159944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 924/1842] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:47.163132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e772b-147c-4978-9ec3-0829b8f50cd9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.163944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.168974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.169843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.170550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.171175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.171878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.173502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.176736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313bf85-55eb-4996-a9b8-7c83432cdc5e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.181858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.182500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.182824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.183108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.184156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.185789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.187822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 919/1843] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:47.190299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.200283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 925/1844] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:47.201953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497e094-fe05-4a08-a350-532b1aac8f98 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.204274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.207487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.208158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.208527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.208808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.209757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.212235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.214799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd37c98-da1f-44d1-b8ef-09ea5e35d016 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.219561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79543bbf-aebf-4c0e-b857-428df13c7bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.220368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543bbf-aebf-4c0e-b857-428df13c7bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.220673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543bbf-aebf-4c0e-b857-428df13c7bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.220954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543bbf-aebf-4c0e-b857-428df13c7bff admin admin] PATH_INFO: `/v3/projects/932be763bf7d4e14a6ec3b2f9ed6edf8/users/49e3e57525db46e1bb0e0e5458fb1632/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.227814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 920/1845] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:47.229420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.239139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79543bbf-aebf-4c0e-b857-428df13c7bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=932be763bf7d4e14a6ec3b2f9ed6edf8, user_id=49e3e57525db46e1bb0e0e5458fb1632, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.240483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98d6f2-9fe1-4cc3-a085-e1c7217565f2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.242256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79543bbf-aebf-4c0e-b857-428df13c7bff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.260276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9c635-cf1e-4329-b56d-67e4bcfb00a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.260690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 926/1846] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:47 2022] PUT /identity/v3/projects/932be763bf7d4e14a6ec3b2f9ed6edf8/users/49e3e57525db46e1bb0e0e5458fb1632/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:47.269768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9c635-cf1e-4329-b56d-67e4bcfb00a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.269768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9c635-cf1e-4329-b56d-67e4bcfb00a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.269768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9c635-cf1e-4329-b56d-67e4bcfb00a5 admin admin] PATH_INFO: `/v3/projects/f443da56f99949e08786f146c794b225/users/d330dd0130194bde94a87041f50f70cc/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.274792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.275327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.276338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.283366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9c635-cf1e-4329-b56d-67e4bcfb00a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f443da56f99949e08786f146c794b225, user_id=d330dd0130194bde94a87041f50f70cc, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.286665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9c635-cf1e-4329-b56d-67e4bcfb00a5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.308712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] MFA Rules not processed for user `49e3e57525db46e1bb0e0e5458fb1632`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:47.309989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.315801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 921/1847] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:47 2022] PUT /identity/v3/projects/f443da56f99949e08786f146c794b225/users/d330dd0130194bde94a87041f50f70cc/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:47.325011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.325301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.325731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.357857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] MFA Rules not processed for user `d330dd0130194bde94a87041f50f70cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:47.359503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.384720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 927/1848] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:47 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:47.389049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.446976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 922/1849] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:47.448602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.452161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.463972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be599a2a-9d2e-43c2-af54-1ae2c34fe63c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.468548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.468907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4283c8d-7001-46d9-9d7e-e437fd80563a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.469173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.469453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.469726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.470395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.472334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.475830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.476777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.477208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.477626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] PATH_INFO: `/v3/projects/af374482606e45cfa8c0de626d71dc90` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.479352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af374482606e45cfa8c0de626d71dc90)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.482515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.500488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 928/1850] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:47 2022] POST /identity/v3/projects => generated 401 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:47.504547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.526683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:47.533633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:47.549972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af374482606e45cfa8c0de626d71dc90', 'request_id': 'req-3aec1ff7-331f-437c-b807-e85e7f37d7f6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.556943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af374482606e45cfa8c0de626d71dc90', 'request_id': 'req-3aec1ff7-331f-437c-b807-e85e7f37d7f6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.563797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af374482606e45cfa8c0de626d71dc90', 'request_id': 'req-3aec1ff7-331f-437c-b807-e85e7f37d7f6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.580114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] The token cache is being invalidate because project af374482606e45cfa8c0de626d71dc90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:47.580331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04fb49-ab07-4779-b7ce-5f3e272a84ff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.580415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.581648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.583452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 923/1851] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:47 2022] DELETE /identity/v3/projects/af374482606e45cfa8c0de626d71dc90 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:47.586820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.588238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.644836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.645823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.646302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.646762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.647628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.649780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.659147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec1ff7-331f-437c-b807-e85e7f37d7f6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.664568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.665862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.666482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.667056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.667737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.670044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.680597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 929/1852] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:47 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:47.681351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.684406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.698933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 924/1853] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:47 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:47.698933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.698933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.710181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a93bcb-d5c9-4331-bc89-a458910e4f3d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.764739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3fea-8fb6-4123-99f2-0d7d15c3a656 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.770143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.770859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.771218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.771499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] PATH_INFO: `/v3/users/6c9e35df8e7f46a395a44a67c302c969` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.772807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c9e35df8e7f46a395a44a67c302c969)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.774871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.780685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.781331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.781609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.781884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.782556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.784501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.813546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 925/1854] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:47 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:47.817923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.829861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2e20d-c952-43f1-a19c-29bd8371307a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.838891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.839550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.839880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.840164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.840876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.842543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.844929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c9e35df8e7f46a395a44a67c302c969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.846019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c9e35df8e7f46a395a44a67c302c969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.855064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c9e35df8e7f46a395a44a67c302c969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.858725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 926/1855] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:47.858878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c9e35df8e7f46a395a44a67c302c969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.858980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.858980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.859558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:47.869415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c9e35df8e7f46a395a44a67c302c969'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:47.873687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29766ca7-3334-419e-b92b-122fd0f98413 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.880318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.881137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.881561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.881963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.882924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:47.884911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:47.892964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 930/1856] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:47 2022] DELETE /identity/v3/users/6c9e35df8e7f46a395a44a67c302c969 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:47.893793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.899808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 927/1857] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:47.903045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:47.908295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:47.914153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308e3d6-6a3f-4cbc-b15d-aac5fe4a0046 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.932452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:DELETE=5,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:47.983033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fd763-62f4-4f86-a97f-11258a42e371 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:47.988336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:47.989147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:47.989499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:47.989866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] PATH_INFO: `/v3/projects/789c1616462249fc859f52ed60d5c8b9/users/985521efe34f4b66b4e8133dbe791b2f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:47.999261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.000497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.001081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.001566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] PATH_INFO: `/v3/projects/21cd75067e6a44af9633bab60b006019/users/d555ae755f7a4605825057a841ea0faf/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.005477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=789c1616462249fc859f52ed60d5c8b9, user_id=985521efe34f4b66b4e8133dbe791b2f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.008330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.024496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cd75067e6a44af9633bab60b006019, user_id=d555ae755f7a4605825057a841ea0faf, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.024696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 931/1858] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:47 2022] PUT /identity/v3/projects/789c1616462249fc859f52ed60d5c8b9/users/985521efe34f4b66b4e8133dbe791b2f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:48.027392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.032956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.046159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 928/1859] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:47 2022] PUT /identity/v3/projects/21cd75067e6a44af9633bab60b006019/users/d555ae755f7a4605825057a841ea0faf/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:48.052079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.118942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab522d-31ae-441c-91cd-ec760221ab6b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.129035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683da2-c472-46f6-8d45-1a37e6852648 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.136968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.137353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.137635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.137916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.138622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.140300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.155122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 929/1860] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:48.156779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.173986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe340db-9f3b-4000-8cdf-bee9614ede12 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.179517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.180136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e84d4-0351-49c7-89f0-e3153e7d0a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.180258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.180561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.180770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e84d4-0351-49c7-89f0-e3153e7d0a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.180882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.181123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e84d4-0351-49c7-89f0-e3153e7d0a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.181624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e84d4-0351-49c7-89f0-e3153e7d0a6e admin admin] PATH_INFO: `/v3/projects/21cd75067e6a44af9633bab60b006019/users/d555ae755f7a4605825057a841ea0faf/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.181914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.183747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.183900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e84d4-0351-49c7-89f0-e3153e7d0a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cd75067e6a44af9633bab60b006019, user_id=d555ae755f7a4605825057a841ea0faf, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.186865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e84d4-0351-49c7-89f0-e3153e7d0a6e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.189272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:48.195416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 932/1861] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:48.200025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.210583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 930/1862] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:48 2022] PUT /identity/v3/projects/21cd75067e6a44af9633bab60b006019/users/d555ae755f7a4605825057a841ea0faf/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:48.212893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.212893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.213103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.216899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105431-60c2-4ebc-98d1-8425b13c9bba admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.244863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] MFA Rules not processed for user `d555ae755f7a4605825057a841ea0faf`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:48.246278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.316997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4c7a5-9f9b-451e-9b2b-d6ff59e44df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.317716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4c7a5-9f9b-451e-9b2b-d6ff59e44df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.317999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4c7a5-9f9b-451e-9b2b-d6ff59e44df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.318278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4c7a5-9f9b-451e-9b2b-d6ff59e44df3 admin admin] PATH_INFO: `/v3/projects/789c1616462249fc859f52ed60d5c8b9/users/985521efe34f4b66b4e8133dbe791b2f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.320294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4c7a5-9f9b-451e-9b2b-d6ff59e44df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=789c1616462249fc859f52ed60d5c8b9, user_id=985521efe34f4b66b4e8133dbe791b2f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.323119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4c7a5-9f9b-451e-9b2b-d6ff59e44df3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.338712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 931/1863] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:48.339280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 933/1864] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:48 2022] PUT /identity/v3/projects/789c1616462249fc859f52ed60d5c8b9/users/985521efe34f4b66b4e8133dbe791b2f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:48.344284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.346723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.347907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.348266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.348613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.361251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58654ce0-a915-4dc0-b8d3-f2ae34d7a89d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.365621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:48.380941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] MFA Rules not processed for user `985521efe34f4b66b4e8133dbe791b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:48.382374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.422683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.423570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.423943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.424288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] PATH_INFO: `/v3/projects/c56843fa38b34a4ba9e710cb962e3064` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.425991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56843fa38b34a4ba9e710cb962e3064)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.428085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.464077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 934/1865] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:48.468486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.469813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.482685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56843fa38b34a4ba9e710cb962e3064', 'request_id': 'req-a12f7dae-8108-4f49-afd2-ace071ceac73'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:48.489639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56843fa38b34a4ba9e710cb962e3064', 'request_id': 'req-a12f7dae-8108-4f49-afd2-ace071ceac73'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:48.497102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56843fa38b34a4ba9e710cb962e3064', 'request_id': 'req-a12f7dae-8108-4f49-afd2-ace071ceac73'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:48.512179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] The token cache is being invalidate because project c56843fa38b34a4ba9e710cb962e3064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:48.512474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:48.513114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:48.514779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 932/1866] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:48 2022] DELETE /identity/v3/projects/c56843fa38b34a4ba9e710cb962e3064 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:48.515590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:48.524969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.540829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a95583-04f4-482b-8f42-36cf7694ab4d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.586106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:48.610109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.610958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.611244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.611521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.616169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.687185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f7dae-8108-4f49-afd2-ace071ceac73 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:48.714203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:48.716967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.718116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.718254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.718781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.723801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccf91a-6a95-4f37-a967-2b248f9a4450 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.726516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.749012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 935/1867] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:48.753111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.753400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.753679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:48.804517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:48.805764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:48.847791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:48.880974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:48.893808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226152b1-e1cf-4b71-afa0-670828e3a281 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:48.915126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 933/1868] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:48 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:48.937390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 936/1869] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:48.950984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:48.951284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:48.951566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.009355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:49.010845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:49.032140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 934/1870] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:49.035796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.035942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.047566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618142de-684d-4526-87f5-3d2c157e9949 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.047785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d4fd1-380c-4c42-b33c-8ede12fc259c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.051276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.052014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.052365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.052640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.053265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.054158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.054466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.054588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:49.054900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.055985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:49.056391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.058392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.072315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 935/1871] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:13:49.076832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.095259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfee21e-bda3-4e54-ab53-2112c88e3e16 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.106279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.106924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.107205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.107599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.108573 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.111385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.153303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.154997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.156139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 936/1872] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:49 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:49.161561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.168995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 937/1873] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:49.241323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584dbc-a64f-4b76-b9af-657b8f32e2da admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.246744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.247376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.247656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.247965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.248744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.250777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.283096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 937/1874] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:49 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:49.285619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.299002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd7f5e-d567-4622-b7f5-eda4fe6c6ca0 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.309839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.310792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.311617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.312209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.313070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.314903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.332254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 938/1875] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:49.333481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.353374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294cf3-95dd-4eca-91b8-a3d446441819 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.359468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebfd63-9f4e-4acc-98f3-8b7d0fce508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.361168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfd63-9f4e-4acc-98f3-8b7d0fce508f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.361168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfd63-9f4e-4acc-98f3-8b7d0fce508f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.361168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfd63-9f4e-4acc-98f3-8b7d0fce508f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.361416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebfd63-9f4e-4acc-98f3-8b7d0fce508f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.363184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebfd63-9f4e-4acc-98f3-8b7d0fce508f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.379268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 938/1876] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:49.383492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.400304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d12bd-1cf9-4a3e-8c2a-aed3332eae7d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.406500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.407273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.407675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.408052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] PATH_INFO: `/v3/projects/1e13c93d4ff4473cb96fa52bd73b4136/users/19e5fd4806d747f5987f86720a498219/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.431060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e13c93d4ff4473cb96fa52bd73b4136, user_id=19e5fd4806d747f5987f86720a498219, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.436231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.439800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:49.454000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 939/1877] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:49 2022] PUT /identity/v3/projects/1e13c93d4ff4473cb96fa52bd73b4136/users/19e5fd4806d747f5987f86720a498219/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:49.465888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.465888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.466108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.518647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] MFA Rules not processed for user `19e5fd4806d747f5987f86720a498219`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:49.522478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:49.604397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 939/1878] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:49.609382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.672758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6a921-f170-46ef-a665-85ee2b021832 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.683870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.683870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.683870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.683870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.684252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.684252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.718219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 940/1879] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:49 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:49.724169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.735946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7134ed-c290-4baa-b886-57869b3d5efa None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.745629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:49.795486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.796136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.796416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.796718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.797342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.799265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.822689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 940/1880] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:49 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:49.826699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.836831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d817b-b7e7-4fa9-8992-d5ac8bae08ea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.842079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.842715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.843020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.843286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.844038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.845637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.858019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 941/1881] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:49.861934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.871960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20781c4-bfb1-4e10-ada1-d6da54a62c99 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.876732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702179a-cee9-4e1d-b2dc-61e3a8ff2198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.877341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702179a-cee9-4e1d-b2dc-61e3a8ff2198 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.877622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702179a-cee9-4e1d-b2dc-61e3a8ff2198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.877895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702179a-cee9-4e1d-b2dc-61e3a8ff2198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.878589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702179a-cee9-4e1d-b2dc-61e3a8ff2198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.880314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702179a-cee9-4e1d-b2dc-61e3a8ff2198 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.889939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 941/1882] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:49.893976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:49.904673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c064-5845-456e-b396-9697fe1b7e5f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:49.910263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:49.910940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.911218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.911487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] PATH_INFO: `/v3/projects/2e11c1353a7b448caefda4f015a3b798/users/5fb7cebd90854d02b774947ab90fe4e0/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.927221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e11c1353a7b448caefda4f015a3b798, user_id=5fb7cebd90854d02b774947ab90fe4e0, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:49.929828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:49.948839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 942/1883] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:49 2022] PUT /identity/v3/projects/2e11c1353a7b448caefda4f015a3b798/users/5fb7cebd90854d02b774947ab90fe4e0/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:49.958848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:49.959023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:49.959335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:49.989485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] MFA Rules not processed for user `5fb7cebd90854d02b774947ab90fe4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:49.990694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:50.055247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 942/1884] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:50.060075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.126017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4398588-eb1e-4666-814c-71474acb128c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.131055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.131683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.131993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.132249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.132935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.134860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.164473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 943/1885] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:50 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:50.169132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.180036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fa957-cae6-43b6-8d0b-5b3c7eade5b8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.247252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.247942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.248231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.248512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.249445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.251670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.279406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 943/1886] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:50 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:50.280814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.291868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8795-c5c3-40f4-8db6-07bfc08202cc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.296451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.297071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.297376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.297656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.298350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.300008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.310781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 944/1887] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:50.314990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.325102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a75f-c766-494f-b745-802c010bd2a9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.329720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.330396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.330681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.330987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] PATH_INFO: `/v3/projects/1e60ed6d770c4e3abe1b4a0cee445ef9/users/3fd40e88ad32470da725f81492f2c220/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.339661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:50.347887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e60ed6d770c4e3abe1b4a0cee445ef9, user_id=3fd40e88ad32470da725f81492f2c220, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.350881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.364619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 944/1888] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:50 2022] PUT /identity/v3/projects/1e60ed6d770c4e3abe1b4a0cee445ef9/users/3fd40e88ad32470da725f81492f2c220/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:50.368695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.382460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4b74f-efcf-4368-b14a-8d55c5ef1769 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.395164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:50.440223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297d30b-935b-41aa-8283-70895d0fd4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.440861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297d30b-935b-41aa-8283-70895d0fd4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.441144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297d30b-935b-41aa-8283-70895d0fd4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.441506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297d30b-935b-41aa-8283-70895d0fd4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.442172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297d30b-935b-41aa-8283-70895d0fd4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.443788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297d30b-935b-41aa-8283-70895d0fd4ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.454026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 945/1889] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:50.458148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.469034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59941024-7ced-4527-9df3-3970d8a0ccab admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.474272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.474940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.475222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.475525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] PATH_INFO: `/v3/projects/1e60ed6d770c4e3abe1b4a0cee445ef9/users/3fd40e88ad32470da725f81492f2c220/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.477678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e60ed6d770c4e3abe1b4a0cee445ef9, user_id=3fd40e88ad32470da725f81492f2c220, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.480358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.490186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 945/1890] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:50 2022] PUT /identity/v3/projects/1e60ed6d770c4e3abe1b4a0cee445ef9/users/3fd40e88ad32470da725f81492f2c220/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:50.501765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.502058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.502334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.530405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] MFA Rules not processed for user `3fd40e88ad32470da725f81492f2c220`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:50.531614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:50.615964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 946/1891] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:50.624823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:50.626561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:50.688080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df907d0-b33d-4ece-a9ac-03d0688055f3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:50.693120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b85d54-ee51-467e-8a1a-853a9fddf6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:50.693718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b85d54-ee51-467e-8a1a-853a9fddf6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:50.694002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b85d54-ee51-467e-8a1a-853a9fddf6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:50.694274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b85d54-ee51-467e-8a1a-853a9fddf6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:50.704707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b85d54-ee51-467e-8a1a-853a9fddf6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:50.706230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b85d54-ee51-467e-8a1a-853a9fddf6f4 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:50.718123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 946/1892] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:51.308626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:51.318735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d5fa-6cf6-476e-a4f7-1312dee0c99d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:51.322998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a81a03-7955-4728-ae4e-f556da20cff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:51.323631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81a03-7955-4728-ae4e-f556da20cff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:51.323941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81a03-7955-4728-ae4e-f556da20cff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:51.324216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81a03-7955-4728-ae4e-f556da20cff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:51.384487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a81a03-7955-4728-ae4e-f556da20cff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:51.385136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a81a03-7955-4728-ae4e-f556da20cff6 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:51.395400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 947/1893] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:54.830854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633c6f-23e1-4f99-82b0-0adda1b1b18b None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:54.832046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633c6f-23e1-4f99-82b0-0adda1b1b18b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:54.832334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633c6f-23e1-4f99-82b0-0adda1b1b18b None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:54.834503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 947/1894] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:13:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:13:54.843579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:54.844115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:54.844314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:54.868490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:54.869802 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:54.904963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:54.968930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 948/1895] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:13:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:54.988462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4633c6f-23e1-4f99-82b0-0adda1b1b18b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:54.990608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f4633c6f-23e1-4f99-82b0-0adda1b1b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:55.013871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4633c6f-23e1-4f99-82b0-0adda1b1b18b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:55.023416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:55.024104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:55.024386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:55.024675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:55.034611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:55.036875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:55.063991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 948/1896] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:55.727602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:55.739080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71515fae-1d7a-4bdc-9be8-d5948f193dcf None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:55.744771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:55.745409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:55.745844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:55.746261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:55.748258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:55.838816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:55.840234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:55.862010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 949/1897] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:56.027700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:56.038030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ada7f-7103-4f7e-9ef1-578a46e8401f service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:56.042279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f688432-902f-4a79-b148-e50bbce9dd76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:56.042970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f688432-902f-4a79-b148-e50bbce9dd76 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:56.043518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f688432-902f-4a79-b148-e50bbce9dd76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:56.043884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f688432-902f-4a79-b148-e50bbce9dd76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:56.104483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f688432-902f-4a79-b148-e50bbce9dd76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:56.106090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f688432-902f-4a79-b148-e50bbce9dd76 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:56.115376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 949/1898] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:56.414962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:56.417215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:56.526223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14d81-f585-4986-aaa4-4c955a85fe16 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:56.533607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:56.534613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:56.535116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:56.535614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] PATH_INFO: `/v3/users/49e3e57525db46e1bb0e0e5458fb1632` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:56.537253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e3e57525db46e1bb0e0e5458fb1632)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:56.540078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:56.627901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e3e57525db46e1bb0e0e5458fb1632'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:56.628790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e3e57525db46e1bb0e0e5458fb1632'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:56.640186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e3e57525db46e1bb0e0e5458fb1632'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:56.642742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e3e57525db46e1bb0e0e5458fb1632'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:56.664492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e3e57525db46e1bb0e0e5458fb1632'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:56.669855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:56.699630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 950/1899] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:56 2022] DELETE /identity/v3/users/49e3e57525db46e1bb0e0e5458fb1632 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:56.704314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:56.704678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:56.705957 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:56.746718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:56.748261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:56.832042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 950/1900] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:56.841779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:56.843182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:56.899319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65ad6f-11fe-4c18-9129-9929dc5d8366 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:56.902898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:56.903522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:56.903831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:56.904108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:56.913481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:56.914914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:56.925056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 951/1901] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:13:57.607085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:57.610315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:57.622393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:57.629542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd69477-bf48-41e7-85a0-6e6c138c6cb8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:57.650013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:57.650013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:57.650013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:57.650013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] PATH_INFO: `/v3/projects/932be763bf7d4e14a6ec3b2f9ed6edf8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:57.650289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] RBAC: Authorizing `identity:delete_project(project_id=932be763bf7d4e14a6ec3b2f9ed6edf8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:57.650289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:57.697211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932be763bf7d4e14a6ec3b2f9ed6edf8', 'request_id': 'req-134bd339-253c-4fc0-87e9-e0b17043247f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.702670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932be763bf7d4e14a6ec3b2f9ed6edf8', 'request_id': 'req-134bd339-253c-4fc0-87e9-e0b17043247f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.708924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932be763bf7d4e14a6ec3b2f9ed6edf8', 'request_id': 'req-134bd339-253c-4fc0-87e9-e0b17043247f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.722352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] The token cache is being invalidate because project 932be763bf7d4e14a6ec3b2f9ed6edf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:57.722639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.723265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.724865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 951/1902] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:57 2022] DELETE /identity/v3/projects/932be763bf7d4e14a6ec3b2f9ed6edf8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:57.729005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:57.741451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f048-f816-4492-8e59-355a71a0f6aa service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:57.825235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:57.826107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:57.826481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:57.826877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] PATH_INFO: `/v3/users/985521efe34f4b66b4e8133dbe791b2f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:57.828212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=985521efe34f4b66b4e8133dbe791b2f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:57.830763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:57.887910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985521efe34f4b66b4e8133dbe791b2f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.890033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985521efe34f4b66b4e8133dbe791b2f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.906528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985521efe34f4b66b4e8133dbe791b2f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.907735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985521efe34f4b66b4e8133dbe791b2f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.920868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d5ce686c-b716-4807-b773-b8f705529b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985521efe34f4b66b4e8133dbe791b2f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:57.949565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 952/1903] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:57 2022] DELETE /identity/v3/users/985521efe34f4b66b4e8133dbe791b2f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:58.479012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:58.480544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:58.505047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bd339-253c-4fc0-87e9-e0b17043247f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:58.614062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:58.614907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:58.615196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:58.615472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] PATH_INFO: `/v3/projects/789c1616462249fc859f52ed60d5c8b9` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:58.618378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=789c1616462249fc859f52ed60d5c8b9)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:58.621775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:58.732546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789c1616462249fc859f52ed60d5c8b9', 'request_id': 'req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:58.742358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789c1616462249fc859f52ed60d5c8b9', 'request_id': 'req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:58.754344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789c1616462249fc859f52ed60d5c8b9', 'request_id': 'req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:58.788308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] The token cache is being invalidate because project 789c1616462249fc859f52ed60d5c8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:58.789077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:58.790511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:58.792571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 952/1904] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:58 2022] DELETE /identity/v3/projects/789c1616462249fc859f52ed60d5c8b9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:13:58.797657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25bbc9-d4c1-4b74-99a3-2b791bbdb149 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:58.798225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25bbc9-d4c1-4b74-99a3-2b791bbdb149 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:58.798531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25bbc9-d4c1-4b74-99a3-2b791bbdb149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:58.811026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:58.830409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ba25bbc9-d4c1-4b74-99a3-2b791bbdb149 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:58.831898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ba25bbc9-d4c1-4b74-99a3-2b791bbdb149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:58.919533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:58.921072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:58.927817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 953/1905] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:58.932197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:58.932689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:58.933035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:58.944844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d78bc-a75b-464c-bab8-e47131eb4c4b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:58.956648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:58.957335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:58.957679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:58.958105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] PATH_INFO: `/v3/users/19e5fd4806d747f5987f86720a498219` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:58.959656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e5fd4806d747f5987f86720a498219)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:58.960813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:58.963048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:58.964145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:58.973173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 954/1906] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:13:58.977305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:58.982976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:58.989217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237041-6ede-4a9d-be49-4a61255abf28 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.000955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.002768 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.003009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.003426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.004586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:13:59.005089 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.007564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.021584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 955/1907] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:13:59.025763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.036891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dfe39-5143-4170-95d2-30c1b2815a23 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.042541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.043268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.043383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e5fd4806d747f5987f86720a498219'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.043799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.044250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.044373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e5fd4806d747f5987f86720a498219'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.045200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.047579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.054412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e5fd4806d747f5987f86720a498219'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.054919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e5fd4806d747f5987f86720a498219'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.066010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3e3b7b3d-e884-4ceb-bd1d-eedeb9c586fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e5fd4806d747f5987f86720a498219'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.091254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 953/1908] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:58 2022] DELETE /identity/v3/users/19e5fd4806d747f5987f86720a498219 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:59.094036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 956/1909] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:59 2022] POST /identity/v3/projects => generated 409 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:59.097203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.097642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.098056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.101448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.103224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.119508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:59.120783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.192313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669eace-d8fc-419f-8503-3d83c89e3c06 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.197659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.198368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.198656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.198933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.199797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.199908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 954/1910] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:59.201882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.214591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.217354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.235815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 957/1911] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:59 2022] POST /identity/v3/users => generated 305 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:59.238929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.251702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3a257-15ce-464f-8a4f-63202c160924 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.258981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.259998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.260455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.260868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.261841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.264113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.276884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 958/1912] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:13:59.280524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.305354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263bb25-657b-47d1-bcb6-77ba51147487 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.307729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40580-1b61-4047-84c5-f66e5ff2ab44 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.314272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.314434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe173f8-576a-4acc-b06f-1df16740f08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.314951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe173f8-576a-4acc-b06f-1df16740f08a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.315212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.315310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe173f8-576a-4acc-b06f-1df16740f08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.315489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe173f8-576a-4acc-b06f-1df16740f08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.315793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.316247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.317038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.318793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.327694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe173f8-576a-4acc-b06f-1df16740f08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.329443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe173f8-576a-4acc-b06f-1df16740f08a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.331838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 959/1913] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:59.335715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.345232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 955/1914] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:13:59.351798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa896f3-e302-478a-95f9-4ec646f85634 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.357652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.358459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.358874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.359389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] PATH_INFO: `/v3/projects/3773885dfc274bc2acfd744c54a5ab6c/users/710d31610c754bbb98cfdc6f521a9c97/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.387071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3773885dfc274bc2acfd744c54a5ab6c, user_id=710d31610c754bbb98cfdc6f521a9c97, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.390397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.413366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 960/1915] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:59 2022] PUT /identity/v3/projects/3773885dfc274bc2acfd744c54a5ab6c/users/710d31610c754bbb98cfdc6f521a9c97/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:59.424959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.425398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.425823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.469980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] MFA Rules not processed for user `710d31610c754bbb98cfdc6f521a9c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:13:59.471459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.502777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:59.573009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 956/1916] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:13:59.578093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.591236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:13:59.671068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1fcd0-2587-4abb-92e4-c02515367ffb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.676004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.676754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.677158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.677988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.678625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.680671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.704621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.706273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.720875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 961/1917] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:13:59 2022] POST /identity/v3/projects => generated 407 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:13:59.724904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.737560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202a472-3b2c-4c35-8c18-9826111cc1d5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.801281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.801990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.802342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.802628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] PATH_INFO: `/v3/projects/1e13c93d4ff4473cb96fa52bd73b4136` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.803850 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e13c93d4ff4473cb96fa52bd73b4136)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.805787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.807851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae3b0c-99dc-4776-a197-8e2a6fb575c7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.813855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.815046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.815736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.816185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.817313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.819986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.847253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 962/1918] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:13:59 2022] POST /identity/v3/users => generated 310 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:13:59.851334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.860778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e13c93d4ff4473cb96fa52bd73b4136', 'request_id': 'req-d134fa8f-00cd-4612-8b27-1b34b9693105'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.868420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e13c93d4ff4473cb96fa52bd73b4136', 'request_id': 'req-d134fa8f-00cd-4612-8b27-1b34b9693105'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.869117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d60216-83b8-4f7d-8268-5f62e89fa4de admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:13:59.876218 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e13c93d4ff4473cb96fa52bd73b4136', 'request_id': 'req-d134fa8f-00cd-4612-8b27-1b34b9693105'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.896314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] The token cache is being invalidate because project 1e13c93d4ff4473cb96fa52bd73b4136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:13:59.896314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.897388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:13:59.899556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 957/1919] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:13:59 2022] DELETE /identity/v3/projects/1e13c93d4ff4473cb96fa52bd73b4136 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:13:59.904200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.906631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.942837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:13:59.943615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:13:59.944247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:13:59.944624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:13:59.945946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:13:59.948043 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:13:59.963153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 963/1920] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:13:59.968771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:13:59.971722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:13:59.983188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:00.003747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d0e8e-e0ea-4c0c-b735-c7f32965cb7c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.011013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.011884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.012264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.012655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] PATH_INFO: `/v3/projects/100ad51e2a5d445db89dbd10a73eedc8/users/fe8bd34b200b444d8615a0b966803e6e/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.018707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134fa8f-00cd-4612-8b27-1b34b9693105 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.030480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.031158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.031441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.031717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] PATH_INFO: `/v3/users/5fb7cebd90854d02b774947ab90fe4e0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.032921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100ad51e2a5d445db89dbd10a73eedc8, user_id=fe8bd34b200b444d8615a0b966803e6e, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.036292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.039266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb7cebd90854d02b774947ab90fe4e0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.041609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.042783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:00.063974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 964/1921] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:13:59 2022] PUT /identity/v3/projects/100ad51e2a5d445db89dbd10a73eedc8/users/fe8bd34b200b444d8615a0b966803e6e/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:00.065458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:00.150289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb7cebd90854d02b774947ab90fe4e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.151117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb7cebd90854d02b774947ab90fe4e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.160142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931fbfe-73e5-43f7-8be8-37fe0f673864 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.160622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb7cebd90854d02b774947ab90fe4e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.161533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb7cebd90854d02b774947ab90fe4e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.166312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.166959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.167246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.167528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.168503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.170892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.180207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-50771288-cf9c-4ae3-abff-155d477ab764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb7cebd90854d02b774947ab90fe4e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.192499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 965/1922] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:00.196570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:00.200256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:00.205893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 958/1923] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:13:59 2022] DELETE /identity/v3/users/5fb7cebd90854d02b774947ab90fe4e0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:00.216585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6cad2-db94-491e-8356-5ff3b2575318 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.279732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.280559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.280916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.281265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] PATH_INFO: `/v3/projects/100ad51e2a5d445db89dbd10a73eedc8/users/fe8bd34b200b444d8615a0b966803e6e/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.283757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100ad51e2a5d445db89dbd10a73eedc8, user_id=fe8bd34b200b444d8615a0b966803e6e, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.286905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.307332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 966/1924] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:00 2022] PUT /identity/v3/projects/100ad51e2a5d445db89dbd10a73eedc8/users/fe8bd34b200b444d8615a0b966803e6e/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:00.313407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.313697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.314041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.366392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] MFA Rules not processed for user `fe8bd34b200b444d8615a0b966803e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:00.369453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:00.463642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 959/1925] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:00.465103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:00.467803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:00.552889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a9244-12bb-4526-b169-065c1438d5d2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.556556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.557188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.557531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.557967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.567432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.568989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.579365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 967/1926] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:00.680068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:00.681095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:00.760252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:00.767716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd94bf-d8fd-4af9-9fb6-51411ca5816d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.774083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.775930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.777122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.777570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] PATH_INFO: `/v3/projects/2e11c1353a7b448caefda4f015a3b798` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.780868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e11c1353a7b448caefda4f015a3b798)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.783589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.800441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:00.810124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa470a7-09f5-43ca-91fe-4341ad103a40 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.816165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.816871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.817218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.817579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.820977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:00.857904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e11c1353a7b448caefda4f015a3b798', 'request_id': 'req-21293351-7111-41a8-9088-3f1afab034d9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.859681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:00.864807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e11c1353a7b448caefda4f015a3b798', 'request_id': 'req-21293351-7111-41a8-9088-3f1afab034d9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.873898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e11c1353a7b448caefda4f015a3b798', 'request_id': 'req-21293351-7111-41a8-9088-3f1afab034d9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.888586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] The token cache is being invalidate because project 2e11c1353a7b448caefda4f015a3b798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:00.889117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.889756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:00.891527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 960/1927] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:00 2022] DELETE /identity/v3/projects/2e11c1353a7b448caefda4f015a3b798 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:00.895253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:00.896888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:00.903750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.904685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:00.916397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 968/1928] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:00.965144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21293351-7111-41a8-9088-3f1afab034d9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:00.970924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:00.971595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:00.971924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:00.972207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] PATH_INFO: `/v3/users/3fd40e88ad32470da725f81492f2c220` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:00.973512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd40e88ad32470da725f81492f2c220)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:00.975578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:01.044440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd40e88ad32470da725f81492f2c220'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.045164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd40e88ad32470da725f81492f2c220'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.053726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd40e88ad32470da725f81492f2c220'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.054659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd40e88ad32470da725f81492f2c220'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.065271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8003097d-8f87-47ae-b0ef-0e04960bdc7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd40e88ad32470da725f81492f2c220'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.094104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 961/1929] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:00 2022] DELETE /identity/v3/users/3fd40e88ad32470da725f81492f2c220 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:01.469677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:01.484239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e782d-5118-41dc-9381-3f67c00a88c9 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:01.561223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:01.561875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:01.562154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:01.562423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] PATH_INFO: `/v3/projects/1e60ed6d770c4e3abe1b4a0cee445ef9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:01.563599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e60ed6d770c4e3abe1b4a0cee445ef9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:01.565753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:01.616919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e60ed6d770c4e3abe1b4a0cee445ef9', 'request_id': 'req-53858000-c149-4a0f-9656-eb6689f094c9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.623375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e60ed6d770c4e3abe1b4a0cee445ef9', 'request_id': 'req-53858000-c149-4a0f-9656-eb6689f094c9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.630027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e60ed6d770c4e3abe1b4a0cee445ef9', 'request_id': 'req-53858000-c149-4a0f-9656-eb6689f094c9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.654481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] The token cache is being invalidate because project 1e60ed6d770c4e3abe1b4a0cee445ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:01.654779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.655411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-53858000-c149-4a0f-9656-eb6689f094c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:01.657512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 969/1930] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:01 2022] DELETE /identity/v3/projects/1e60ed6d770c4e3abe1b4a0cee445ef9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:01.663941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:01.664296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:01.664644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:01.676064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:01.696008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:01.697221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:01.783575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 962/1931] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:01.791573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:01.791900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:01.792205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:01.816028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:01.817309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:01.826552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 970/1932] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:01.831011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:01.841534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45e0e4-1649-421f-adf9-b56b418aa1c6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:01.846113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:01.846744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:01.847028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:01.847298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:01.848169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:01.848478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:01.850168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:01.865727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 963/1933] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:01.867142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:01.876881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7e71-81bc-4df6-9ae2-c8cd90e01c26 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:01.881609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:01.882258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:01.882545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:01.882824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:01.883513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:01.885478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:01.920728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 971/1934] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:01 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:01.921733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:01.932278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91d587-0aba-4e27-91e0-df6173571f13 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:01.991165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:01.991866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:01.992154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:01.992432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:01.993140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:01.995095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.017276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 964/1935] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:01 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:02.021704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.032390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da9579-2981-4eaa-ade5-56e8257597c1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.037402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.038049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.038334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.038611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.039358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.041525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.053051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 972/1936] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:02.057408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.067904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7613a99-fc44-4bf9-9f28-173e5f1e2689 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.072768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abfd7c-5f22-4095-9854-7c27024e2b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.073400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abfd7c-5f22-4095-9854-7c27024e2b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.073682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abfd7c-5f22-4095-9854-7c27024e2b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.073965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abfd7c-5f22-4095-9854-7c27024e2b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.074669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abfd7c-5f22-4095-9854-7c27024e2b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.076312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abfd7c-5f22-4095-9854-7c27024e2b95 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.088560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 965/1937] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:02.093059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.103124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcadb8-c5c5-48e6-a55b-e1607c2ccb6b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.107617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.108316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.108611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.108886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] PATH_INFO: `/v3/projects/ba00c68c3dad425eb307d04649e1561f/users/6bb570d841404c91bc264fca9fada0da/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.122731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:02.126278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba00c68c3dad425eb307d04649e1561f, user_id=6bb570d841404c91bc264fca9fada0da, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.129532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.145964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 973/1938] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:02 2022] PUT /identity/v3/projects/ba00c68c3dad425eb307d04649e1561f/users/6bb570d841404c91bc264fca9fada0da/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:02.158647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.158986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.159269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.189877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] MFA Rules not processed for user `6bb570d841404c91bc264fca9fada0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:02.191106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:02.252451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:02.260080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 966/1939] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:02.261078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.331759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e91d5-9f68-4fca-95c7-2fbed939a2ee admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.337556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.338519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.338800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.339249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.339979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.341877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.371636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 974/1940] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:02 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:02.375723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.386742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c6d9f-45eb-4990-880e-e871cd6b33fc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.448536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.449112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.449389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.449657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.450324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.452411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.478670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 967/1941] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:02 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:02.483009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.494118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e083d-0c76-4ba8-bafc-25e18d8e49f3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.498725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.499358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.499639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.499952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.500654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.502779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.514104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 975/1942] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:02.518304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.528810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab5dc-d087-4ba3-8a88-493bbe432455 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.534378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.535103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.535381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.535715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] PATH_INFO: `/v3/projects/07d259120ddf4af1b1286744b54b1568/users/d61ad30aba8b4e12afdfb38e959a2d48/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.557992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d259120ddf4af1b1286744b54b1568, user_id=d61ad30aba8b4e12afdfb38e959a2d48, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.560920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.584388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 968/1943] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:02 2022] PUT /identity/v3/projects/07d259120ddf4af1b1286744b54b1568/users/d61ad30aba8b4e12afdfb38e959a2d48/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:02.592913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.603049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceeed2-35e2-40ac-b928-83d73e22e8d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.663494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c748b13-71c7-4702-86b2-b01f5dbc0b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.664468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c748b13-71c7-4702-86b2-b01f5dbc0b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.664800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c748b13-71c7-4702-86b2-b01f5dbc0b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.666167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c748b13-71c7-4702-86b2-b01f5dbc0b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.666277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c748b13-71c7-4702-86b2-b01f5dbc0b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.667933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c748b13-71c7-4702-86b2-b01f5dbc0b0d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.683055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 976/1944] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:02.684577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.695335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c85e7-9d6a-42dc-b54b-e1afba45fdec admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.700858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.700858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.701193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.701272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] PATH_INFO: `/v3/projects/07d259120ddf4af1b1286744b54b1568/users/d61ad30aba8b4e12afdfb38e959a2d48/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.703351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d259120ddf4af1b1286744b54b1568, user_id=d61ad30aba8b4e12afdfb38e959a2d48, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.706016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.718116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 969/1945] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:02 2022] PUT /identity/v3/projects/07d259120ddf4af1b1286744b54b1568/users/d61ad30aba8b4e12afdfb38e959a2d48/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:02.728470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.728762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.729045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.765278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] MFA Rules not processed for user `d61ad30aba8b4e12afdfb38e959a2d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:02.766744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:02.813501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:02.837475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 977/1946] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:02.848786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.851451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:02.911034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a9663-1d78-447e-97e6-ec757d9a1577 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:02.914866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:02.915917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:02.916202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:02.916488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:02.926039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:02.927562 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:02.937396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 970/1947] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:02.991291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:02.992813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:03.005104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270d02f-70f4-4069-bf00-bbcda0c81e3a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:03.009295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c0a93-f856-44ab-9357-107d7af269c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:03.009946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a93-f856-44ab-9357-107d7af269c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:03.010263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a93-f856-44ab-9357-107d7af269c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:03.010608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a93-f856-44ab-9357-107d7af269c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:03.068435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c0a93-f856-44ab-9357-107d7af269c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:03.069878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c0a93-f856-44ab-9357-107d7af269c1 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:03.082668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 978/1948] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:04.286353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:04.288585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:04.372180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:04.378203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0a19-777d-4147-abf6-0683a967c06f service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:04.384695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:04.385344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:04.385622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:04.385897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] PATH_INFO: `/v3/users/d330dd0130194bde94a87041f50f70cc` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:04.387051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d330dd0130194bde94a87041f50f70cc)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:04.389012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:04.437495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd330dd0130194bde94a87041f50f70cc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:04.438166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd330dd0130194bde94a87041f50f70cc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:04.445112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd330dd0130194bde94a87041f50f70cc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:04.445797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd330dd0130194bde94a87041f50f70cc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:04.453686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd330dd0130194bde94a87041f50f70cc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:04.471900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 971/1949] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:04 2022] DELETE /identity/v3/users/d330dd0130194bde94a87041f50f70cc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:04.476748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:04.477228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:04.477320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:04.496337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:04.497543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:04.557071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 979/1950] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:04.569364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:04.570930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:04.631337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82118f-9044-4340-afa1-ba98fd37333e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:04.634998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:04.635711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:04.635930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:04.636201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:04.646166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:04.647873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:04.657820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 972/1951] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:04.935679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:04.937249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:04.951174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebb6aa-593e-4087-9969-8aaba6476b13 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:04.956641 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:04.957306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:04.957588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:04.957861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] PATH_INFO: `/v3/projects/f443da56f99949e08786f146c794b225` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:04.959037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f443da56f99949e08786f146c794b225)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:04.962266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:05.017274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f443da56f99949e08786f146c794b225', 'request_id': 'req-74b0d0aa-6d27-41b1-be1a-bba32428e897'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.022957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f443da56f99949e08786f146c794b225', 'request_id': 'req-74b0d0aa-6d27-41b1-be1a-bba32428e897'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.029314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f443da56f99949e08786f146c794b225', 'request_id': 'req-74b0d0aa-6d27-41b1-be1a-bba32428e897'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.043355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] The token cache is being invalidate because project f443da56f99949e08786f146c794b225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:05.043733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.044468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.046112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 980/1952] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:04 2022] DELETE /identity/v3/projects/f443da56f99949e08786f146c794b225 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:05.050355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:05.060564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4124357-31d2-4ae4-9ab0-7d1a4e7e0698 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:05.117205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:05.117938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:05.118234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:05.118475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] PATH_INFO: `/v3/users/d555ae755f7a4605825057a841ea0faf` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:05.119611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d555ae755f7a4605825057a841ea0faf)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:05.121544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:05.142995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:05.171782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd555ae755f7a4605825057a841ea0faf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.172449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd555ae755f7a4605825057a841ea0faf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.180288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd555ae755f7a4605825057a841ea0faf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.183021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd555ae755f7a4605825057a841ea0faf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.192557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f426234-0560-4d80-8152-c95ffd231f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd555ae755f7a4605825057a841ea0faf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.212519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 973/1953] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:05 2022] DELETE /identity/v3/users/d555ae755f7a4605825057a841ea0faf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:05.548320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:05.550401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:05.582359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0d0aa-6d27-41b1-be1a-bba32428e897 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:05.588440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:05.641894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:05.642675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:05.643012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:05.643348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] PATH_INFO: `/v3/projects/21cd75067e6a44af9633bab60b006019` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:05.644565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cd75067e6a44af9633bab60b006019)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:05.646532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:05.696050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cd75067e6a44af9633bab60b006019', 'request_id': 'req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.701426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cd75067e6a44af9633bab60b006019', 'request_id': 'req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.707579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cd75067e6a44af9633bab60b006019', 'request_id': 'req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.723260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] The token cache is being invalidate because project 21cd75067e6a44af9633bab60b006019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:05.723543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.724292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-813b9ed9-ab0f-43fb-8a73-9af7f4ec29cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:05.725918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 981/1954] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:05 2022] DELETE /identity/v3/projects/21cd75067e6a44af9633bab60b006019 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:05.732429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:05.732899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:05.732986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:05.759537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:05.760785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:05.824937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 974/1955] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:05.837450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:05.837738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:05.838035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:05.860581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:05.861823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:05.871811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 982/1956] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:05.876472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:05.886756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396775e-ace4-4a1c-9e6c-28624b49db0e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:05.892482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:05.893102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:05.893381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:05.893653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:05.894354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:05.894656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:05.896361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:05.911727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 975/1957] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:05.912936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:05.922781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b737d-83e4-4483-8248-db6a2591a467 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:05.928049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:05.928667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:05.928944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:05.929331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:05.930010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:05.931966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:05.961421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 983/1958] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:05 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:05.965698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:05.976112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ea967-323a-4b95-9857-5ffe5d68a8f2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:06.037547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:06.038200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:06.038452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:06.038721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:06.039390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:06.041731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:06.065098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 976/1959] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:05 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:06.069294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:06.079467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e077817-f443-4654-8778-b7c6b1c62b64 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:06.084188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:06.084812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:06.085088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:06.085354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:06.086032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:06.087619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:06.098300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 984/1960] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:06.102317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:06.112811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41332fb-6fde-4b28-83a5-c0da7a422f6b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:06.118598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3fcbf-8490-48c7-bb89-bf54f552fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:06.119219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3fcbf-8490-48c7-bb89-bf54f552fd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:06.119517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3fcbf-8490-48c7-bb89-bf54f552fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:06.119804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3fcbf-8490-48c7-bb89-bf54f552fd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:06.120504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3fcbf-8490-48c7-bb89-bf54f552fd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:06.122100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3fcbf-8490-48c7-bb89-bf54f552fd9d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:06.132995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 977/1961] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:06.137085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:06.147419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b1d60-a67b-4ef2-a21b-97a7adb3910f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:06.152340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:06.152987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:06.153263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:06.153535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] PATH_INFO: `/v3/projects/569b04fa1a2b4f12befe8f802bdc3713/users/b7cb10607145486890a754ae36ca352b/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:06.168803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=569b04fa1a2b4f12befe8f802bdc3713, user_id=b7cb10607145486890a754ae36ca352b, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:06.171445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:06.185647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 985/1962] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:06 2022] PUT /identity/v3/projects/569b04fa1a2b4f12befe8f802bdc3713/users/b7cb10607145486890a754ae36ca352b/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:06.196907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:06.197187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:06.197451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:06.204791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:06.228536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] MFA Rules not processed for user `b7cb10607145486890a754ae36ca352b`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:06.229749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:06.295804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 978/1963] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:06.307459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:06.309386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:06.355962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:06.374862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0429a08-f142-4c6a-b26d-967a32567f02 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:06.378680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3ca8c-d4b5-4ab7-b740-7dbc67156a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:06.379300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3ca8c-d4b5-4ab7-b740-7dbc67156a47 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:06.379578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3ca8c-d4b5-4ab7-b740-7dbc67156a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:06.379882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3ca8c-d4b5-4ab7-b740-7dbc67156a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:06.388213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3ca8c-d4b5-4ab7-b740-7dbc67156a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:06.389651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3ca8c-d4b5-4ab7-b740-7dbc67156a47 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:06.398458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 986/1964] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:08.655383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:08.656875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:08.733681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e32a4d-ef91-41f0-861e-23e29755b631 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:08.739023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:08.740054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:08.740449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:08.740919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] PATH_INFO: `/v3/users/6bb570d841404c91bc264fca9fada0da` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:08.742412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb570d841404c91bc264fca9fada0da)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:08.744752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:08.797175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb570d841404c91bc264fca9fada0da'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:08.797823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb570d841404c91bc264fca9fada0da'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:08.805224 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb570d841404c91bc264fca9fada0da'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:08.805866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb570d841404c91bc264fca9fada0da'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:08.814681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb570d841404c91bc264fca9fada0da'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:08.837412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 979/1965] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:08 2022] DELETE /identity/v3/users/6bb570d841404c91bc264fca9fada0da => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:08.842017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:08.842492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:08.842628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:08.861040 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:08.862223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:08.922740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 987/1966] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:08.932582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:08.934282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:08.948089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:08.994173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24442b9-2b70-44da-8f11-2f18a7db979d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:09.002937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:09.003653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:09.003995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:09.004271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:09.014784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:09.016297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:09.026522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 980/1967] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:09.552157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:09.556761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:09.603261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7068b7d-de1d-4b79-a3c7-6b5192769de9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:09.608299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:09.608942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:09.609645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:09.610059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] PATH_INFO: `/v3/projects/ba00c68c3dad425eb307d04649e1561f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:09.611407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba00c68c3dad425eb307d04649e1561f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:09.613663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:09.663724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba00c68c3dad425eb307d04649e1561f', 'request_id': 'req-05d4287d-42b7-4679-a58f-35d5cb083887'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.670271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba00c68c3dad425eb307d04649e1561f', 'request_id': 'req-05d4287d-42b7-4679-a58f-35d5cb083887'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.676780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba00c68c3dad425eb307d04649e1561f', 'request_id': 'req-05d4287d-42b7-4679-a58f-35d5cb083887'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.690800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] The token cache is being invalidate because project ba00c68c3dad425eb307d04649e1561f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:09.690888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.691499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.692162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:09.693024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 988/1968] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:09 2022] DELETE /identity/v3/projects/ba00c68c3dad425eb307d04649e1561f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:09.693925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:09.696783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:09.698077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:09.706940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaddc5b-6f41-4a5c-8e6c-3e875fd18ef4 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:09.777165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:09.777885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:09.778172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:09.778452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] PATH_INFO: `/v3/users/b7cb10607145486890a754ae36ca352b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:09.779675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cb10607145486890a754ae36ca352b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:09.782077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:09.782990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4287d-42b7-4679-a58f-35d5cb083887 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:09.791589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:09.792421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:09.792773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:09.793141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] PATH_INFO: `/v3/users/d61ad30aba8b4e12afdfb38e959a2d48` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:09.794713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61ad30aba8b4e12afdfb38e959a2d48)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:09.797095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:09.803243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:09.854958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cb10607145486890a754ae36ca352b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.855744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cb10607145486890a754ae36ca352b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.864194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cb10607145486890a754ae36ca352b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.864798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cb10607145486890a754ae36ca352b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.869023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61ad30aba8b4e12afdfb38e959a2d48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.870034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61ad30aba8b4e12afdfb38e959a2d48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.874640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1785d59d-2fe7-4eea-8fa2-26da833f2806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cb10607145486890a754ae36ca352b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.884663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61ad30aba8b4e12afdfb38e959a2d48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.886746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61ad30aba8b4e12afdfb38e959a2d48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.894124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 981/1969] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:09 2022] DELETE /identity/v3/users/b7cb10607145486890a754ae36ca352b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:09.897392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61ad30aba8b4e12afdfb38e959a2d48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:09.899900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:09.900570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:09.900857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:09.922722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:09.924247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:09.935700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 989/1970] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:09 2022] DELETE /identity/v3/users/d61ad30aba8b4e12afdfb38e959a2d48 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:09.964584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:10.008445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 982/1971] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:10.022531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:10.025161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.102831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed747360-9bdb-4fc9-9f85-79298c647bad admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:10.108408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:10.108962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.109473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.109585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.193635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:10.195250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:10.204875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:10.205882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 990/1972] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:10.206421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.221091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16f2b5-1461-4ba2-94ae-33a90504f800 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:10.226968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:10.227878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.228211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.228488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] PATH_INFO: `/v3/users/188226af2d2c463ebae7afd8d9caf4c1` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.229880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188226af2d2c463ebae7afd8d9caf4c1)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:10.231569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:10.281387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:10.283062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.298434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70637a-7969-4f8c-9b8e-df257860b907 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:10.298623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188226af2d2c463ebae7afd8d9caf4c1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.299594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188226af2d2c463ebae7afd8d9caf4c1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.305305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:10.306295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.306801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.307264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] PATH_INFO: `/v3/projects/07d259120ddf4af1b1286744b54b1568` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.309473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d259120ddf4af1b1286744b54b1568)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:10.310654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188226af2d2c463ebae7afd8d9caf4c1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.312536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:10.312992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188226af2d2c463ebae7afd8d9caf4c1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.323604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11329322-19ac-4c84-a821-0e1961d64a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188226af2d2c463ebae7afd8d9caf4c1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.382765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 983/1973] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:10 2022] DELETE /identity/v3/users/188226af2d2c463ebae7afd8d9caf4c1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:10.386521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.386831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.387115 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.392927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:10.424833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d259120ddf4af1b1286744b54b1568', 'request_id': 'req-21c6f325-f917-4163-a928-8f185ba765ba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.439456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:10.440829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.445082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d259120ddf4af1b1286744b54b1568', 'request_id': 'req-21c6f325-f917-4163-a928-8f185ba765ba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.456642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d259120ddf4af1b1286744b54b1568', 'request_id': 'req-21c6f325-f917-4163-a928-8f185ba765ba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.478439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] The token cache is being invalidate because project 07d259120ddf4af1b1286744b54b1568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:10.479034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.481027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-21c6f325-f917-4163-a928-8f185ba765ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:10.489062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 991/1974] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:10 2022] DELETE /identity/v3/projects/07d259120ddf4af1b1286744b54b1568 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:10.490998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5c1ac-1d75-4ee1-8966-30bc5a1e5706 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.491929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5c1ac-1d75-4ee1-8966-30bc5a1e5706 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.492516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5c1ac-1d75-4ee1-8966-30bc5a1e5706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.543845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-04b5c1ac-1d75-4ee1-8966-30bc5a1e5706 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:10.546512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-04b5c1ac-1d75-4ee1-8966-30bc5a1e5706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.596289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 984/1975] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:10.622016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:10.625798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.668879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 992/1976] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:10.672098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.672693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.672831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.750318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:10.753207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:10.779844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:10.792350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8eab2-eae8-48a4-bcac-ea0d6148a108 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:10.803020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 993/1977] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:10.803265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:10.803265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.812294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.812586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.812972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:10.844452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:10.844452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:10.863896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3618-d8ee-42dc-98af-3f93fed9dcb8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:10.864110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 985/1978] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:10.881963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:10.882616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.888405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.888733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.890084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:10.890395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:10.892061 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:10.921894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 994/1979] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:10.926709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:10.954763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6d79e-23eb-4c0c-b376-5f9aa4707b7a service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:10.971112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:10.978164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:10.978757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:10.981008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:10.984301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.002086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.046653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.050189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.051130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 986/1980] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:10 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:11.057458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.070855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dc01-4c91-47f1-bb55-58f206ebd3f2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.145691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189a908-5306-48df-8b15-cc5ad0bf1e14 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.149255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.149948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.150239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.150519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] PATH_INFO: `/v3/users/710d31610c754bbb98cfdc6f521a9c97` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.151801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.152573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710d31610c754bbb98cfdc6f521a9c97)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.152739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.153111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.153481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.154241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.155529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.156440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.221288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 987/1981] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:11 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:11.229776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.242243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:11.247460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730aea6-3e1c-4032-bc0c-4accc20bede1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.256521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710d31610c754bbb98cfdc6f521a9c97'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.256970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.257824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710d31610c754bbb98cfdc6f521a9c97'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.260038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.262212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.262522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.263345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.265081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.277282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710d31610c754bbb98cfdc6f521a9c97'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.283483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 988/1982] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:11.284280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710d31610c754bbb98cfdc6f521a9c97'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.289719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.290861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.312378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a0a89-2756-4913-9035-c63ac157c39c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.313812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710d31610c754bbb98cfdc6f521a9c97'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.318359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.319005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.319290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.319565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.320504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.322194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.340070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 989/1983] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:11.341670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.344370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.351504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 995/1984] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:11 2022] DELETE /identity/v3/users/710d31610c754bbb98cfdc6f521a9c97 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:11.351504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.360750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab4f51-d29b-488e-b84e-a646e61a834f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.427292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.428014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.428299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.428575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] PATH_INFO: `/v3/projects/c74c162641f94714aedb76d4b07585b1` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.430054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74c162641f94714aedb76d4b07585b1)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.430700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3bda-9e05-43fd-9b3b-39ca311cc7f1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.432062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.436287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e05b31-cf6d-4724-ae2f-88b615f150a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.436990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e05b31-cf6d-4724-ae2f-88b615f150a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.437274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e05b31-cf6d-4724-ae2f-88b615f150a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.437549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e05b31-cf6d-4724-ae2f-88b615f150a1 admin admin] PATH_INFO: `/v3/projects/fa4ebdb7cabf464cb9f6bddddb37cdc5/users/38283e25ba9d42ed91d8da2d42269bb0/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.456751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e05b31-cf6d-4724-ae2f-88b615f150a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4ebdb7cabf464cb9f6bddddb37cdc5, user_id=38283e25ba9d42ed91d8da2d42269bb0, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.459269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e05b31-cf6d-4724-ae2f-88b615f150a1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.475170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 996/1985] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:11 2022] PUT /identity/v3/projects/fa4ebdb7cabf464cb9f6bddddb37cdc5/users/38283e25ba9d42ed91d8da2d42269bb0/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:11.476545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.476841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.477119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.480212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:11.497594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74c162641f94714aedb76d4b07585b1', 'request_id': 'req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.500773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:11.501997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.504864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74c162641f94714aedb76d4b07585b1', 'request_id': 'req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.511615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74c162641f94714aedb76d4b07585b1', 'request_id': 'req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.528481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] The token cache is being invalidate because project c74c162641f94714aedb76d4b07585b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:11.529116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.531714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bc93f80-9526-497b-9fc7-3b72b9d2f21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.535153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 990/1986] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:11 2022] DELETE /identity/v3/projects/c74c162641f94714aedb76d4b07585b1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:11.536828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.537312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.537763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.577716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] MFA Rules not processed for user `38283e25ba9d42ed91d8da2d42269bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:11.578591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.584486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 997/1987] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:11.585692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.588128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.603302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e70c5d-3133-4310-93ce-6d4b358fba43 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.612658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.613333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.613618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.613893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] PATH_INFO: `/v3/users/fdf588b1ca53416cbd3d352aa8397e8d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.618903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf588b1ca53416cbd3d352aa8397e8d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.620906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.666820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 991/1988] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:11.669356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.670678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.674562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:11.689987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fc5a5-37ca-4c60-baf5-2a824da46295 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.696749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.697222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf588b1ca53416cbd3d352aa8397e8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.697478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.697763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.698042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] PATH_INFO: `/v3/projects/569b04fa1a2b4f12befe8f802bdc3713` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.698497 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf588b1ca53416cbd3d352aa8397e8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.699264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=569b04fa1a2b4f12befe8f802bdc3713)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.701490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.705920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf588b1ca53416cbd3d352aa8397e8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.706597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf588b1ca53416cbd3d352aa8397e8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.716296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf588b1ca53416cbd3d352aa8397e8d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.737545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 998/1989] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:11 2022] DELETE /identity/v3/users/fdf588b1ca53416cbd3d352aa8397e8d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:11.738239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.741718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.762488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569b04fa1a2b4f12befe8f802bdc3713', 'request_id': 'req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.771978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569b04fa1a2b4f12befe8f802bdc3713', 'request_id': 'req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.780662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569b04fa1a2b4f12befe8f802bdc3713', 'request_id': 'req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.801844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] The token cache is being invalidate because project 569b04fa1a2b4f12befe8f802bdc3713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:11.802563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.804162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:11.806705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 992/1990] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:11 2022] DELETE /identity/v3/projects/569b04fa1a2b4f12befe8f802bdc3713 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:11.807990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.815071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.825137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bcae5-514b-4972-a074-a436c8e1b834 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.890507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:11.923976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffcabe-8877-49e8-bdc8-4f442081b0cf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:11.926642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.931530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.931859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.932171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.933019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:11.933682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.933974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.934240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.934860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:11.934984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.937915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.950613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:11.952491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403595b9-c51b-411a-8f29-732f56bbf401 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:11.964742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 999/1991] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:11.966578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09ae04-8885-497c-b925-d8deffb9b104 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:11.966875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09ae04-8885-497c-b925-d8deffb9b104 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:11.967153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09ae04-8885-497c-b925-d8deffb9b104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:11.976253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 993/1992] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:11 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:11.978293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:11.992144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0c09ae04-8885-497c-b925-d8deffb9b104 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:11.993494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0c09ae04-8885-497c-b925-d8deffb9b104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.108420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0af88-f8c5-48a4-b5ea-ba67dba9ae67 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.111261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1000/1993] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:12.122662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.123115 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.123525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.123830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.124166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.124464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.124617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.125204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.128146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.156590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:12.158465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.163407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 994/1994] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:11 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:12.167604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.173873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1001/1995] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:12.181222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.183338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.187458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc96ca3-f6aa-40aa-96b9-c67c3b418324 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.191237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:12.196837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.198157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.198157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.198157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.198739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.200813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.200954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b711f6a-9ec7-45d3-8cc8-2b183630e509 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.210022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.211134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.212123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.213008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] PATH_INFO: `/v3/projects/901f6b595ad241ad9276489a2769ded1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.214566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] RBAC: Authorizing `identity:delete_project(project_id=901f6b595ad241ad9276489a2769ded1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.216812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.248662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 995/1996] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:12.250365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.265749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97669d36-910e-49ba-966a-17e534530b03 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.290415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.290757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901f6b595ad241ad9276489a2769ded1', 'request_id': 'req-3ddc67d2-58bb-40e2-bb90-8e20f908687a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.293553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.293553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.294332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.296220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:12.296536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.298260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.318374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901f6b595ad241ad9276489a2769ded1', 'request_id': 'req-3ddc67d2-58bb-40e2-bb90-8e20f908687a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.318556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 996/1997] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:12.318556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.324506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901f6b595ad241ad9276489a2769ded1', 'request_id': 'req-3ddc67d2-58bb-40e2-bb90-8e20f908687a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.329446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf261e41-84bb-4af0-a3ae-e660c9815727 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.344592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] The token cache is being invalidate because project 901f6b595ad241ad9276489a2769ded1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:12.345393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.346083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.348045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1002/1998] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:12 2022] DELETE /identity/v3/projects/901f6b595ad241ad9276489a2769ded1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:12.348926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.352671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.416536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57fb99-863a-4c84-a38c-8d50c5ea2694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.417235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57fb99-863a-4c84-a38c-8d50c5ea2694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.417519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57fb99-863a-4c84-a38c-8d50c5ea2694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.417795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57fb99-863a-4c84-a38c-8d50c5ea2694 admin admin] PATH_INFO: `/v3/projects/b4e045b431214806a0745389172ff308/users/3fabcc997db245c39f9c586250905df1/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.436244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57fb99-863a-4c84-a38c-8d50c5ea2694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e045b431214806a0745389172ff308, user_id=3fabcc997db245c39f9c586250905df1, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.439024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc67d2-58bb-40e2-bb90-8e20f908687a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.440609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57fb99-863a-4c84-a38c-8d50c5ea2694 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.447408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.448719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.448719 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.448975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.449860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.452609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.468958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 997/1999] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:12 2022] PUT /identity/v3/projects/b4e045b431214806a0745389172ff308/users/3fabcc997db245c39f9c586250905df1/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:12.472923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.473818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.474107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.499145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:12.500222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1003/2000] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:12 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:12.500681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.501931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.505202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.508051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:12.572134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 998/2001] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:12.572951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.574234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.578500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd707-fcb0-4a49-8086-53b46739cba1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.583327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.584100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.584449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.584914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] PATH_INFO: `/v3/projects/3773885dfc274bc2acfd744c54a5ab6c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.585989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdaf6be-a5b8-4869-ba4b-3ed09c274d0c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.586277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3773885dfc274bc2acfd744c54a5ab6c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.588307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.591564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.592213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.592483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.592755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.593450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.595051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.602647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:12.606116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 999/2002] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:12.607016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.619280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e56fc-3776-4373-9370-f9685664216a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.624535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d748bf-ded7-4170-b347-5846cb682ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.625200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d748bf-ded7-4170-b347-5846cb682ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.625548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d748bf-ded7-4170-b347-5846cb682ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.625796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d748bf-ded7-4170-b347-5846cb682ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.626488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d748bf-ded7-4170-b347-5846cb682ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.628540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d748bf-ded7-4170-b347-5846cb682ed5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.656362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3773885dfc274bc2acfd744c54a5ab6c', 'request_id': 'req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.662791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3773885dfc274bc2acfd744c54a5ab6c', 'request_id': 'req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.672062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3773885dfc274bc2acfd744c54a5ab6c', 'request_id': 'req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.678433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1000/2003] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:12 2022] POST /identity/v3/users => generated 292 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:12.680083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.680461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.680869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.689580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] The token cache is being invalidate because project 3773885dfc274bc2acfd744c54a5ab6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:12.689883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.690512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:12.693445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1004/2004] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:12 2022] DELETE /identity/v3/projects/3773885dfc274bc2acfd744c54a5ab6c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:12.694534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.699743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.703901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:12.705111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.778327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1001/2005] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:12.779320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.782104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0939c-6f59-43f4-bcd1-6ea34d31bf5d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.783815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.791203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.791936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.792222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.792601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] PATH_INFO: `/v3/projects/b4e045b431214806a0745389172ff308/users/3fabcc997db245c39f9c586250905df1/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.794644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e045b431214806a0745389172ff308, user_id=3fabcc997db245c39f9c586250905df1, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.797379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.798645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cca91-1d27-499f-a77e-b8336a2ab943 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.805644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.806291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.806582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.806861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.808830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.809741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1005/2006] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:12 2022] PUT /identity/v3/projects/b4e045b431214806a0745389172ff308/users/3fabcc997db245c39f9c586250905df1/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:12.810595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.812196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:12.814304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.832925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1002/2007] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:12.834050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:12.861732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244000dc-9140-44e8-8ed0-b2f78a3d5e70 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.909843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1bda2-238b-474f-9a2f-47f79093b4c2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:12.923277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.924032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.924322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.924598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] PATH_INFO: `/v3/users/fe8bd34b200b444d8615a0b966803e6e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.930189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8bd34b200b444d8615a0b966803e6e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.930784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:12.931090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.931383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.931994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.932189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.932706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:12.933075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:12.934758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf00a0-a495-4e6b-a0e4-a2d76b60d686 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:12.952367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1003/2008] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:12.952367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021640cd-19e5-4318-8022-2786046456ea None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:12.952367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021640cd-19e5-4318-8022-2786046456ea None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:12.952367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021640cd-19e5-4318-8022-2786046456ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:12.975207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:12.993620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:13.007111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-021640cd-19e5-4318-8022-2786046456ea None None] MFA Rules not processed for user `3fabcc997db245c39f9c586250905df1`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:13.009248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-021640cd-19e5-4318-8022-2786046456ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:13.023792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8bd34b200b444d8615a0b966803e6e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:13.024675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8bd34b200b444d8615a0b966803e6e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:13.032238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8bd34b200b444d8615a0b966803e6e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:13.032804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8bd34b200b444d8615a0b966803e6e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:13.043002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8bd34b200b444d8615a0b966803e6e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:13.062968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1006/2009] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:12 2022] DELETE /identity/v3/users/fe8bd34b200b444d8615a0b966803e6e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:13.063883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.065229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:13.108471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1004/2010] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:13.110771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021640cd-19e5-4318-8022-2786046456ea None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.117725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-021640cd-19e5-4318-8022-2786046456ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:13.142099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021640cd-19e5-4318-8022-2786046456ea None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.149993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.151757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.152078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.152361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.153736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.155979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.156929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f00311-90d1-426b-b785-355e7704ee46 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.162953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.164448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.164877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.166224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.167077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.170563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.183077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1007/2011] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:13.187145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.188943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:13.197211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1005/2012] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:13 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:13.197211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.285886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bb6eb-edca-4d54-b299-b5a66765b14b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.292158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.293741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.294211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.294634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] PATH_INFO: `/v3/projects/45fbc498a16b4e5e94f7e59bbb44e9dd/users/07e98dab752748f39e59562f37384524/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.297548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22512b-c225-4248-9514-55d938774712 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.312876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fbc498a16b4e5e94f7e59bbb44e9dd, user_id=07e98dab752748f39e59562f37384524, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.316516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.338448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1006/2013] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:13 2022] PUT /identity/v3/projects/45fbc498a16b4e5e94f7e59bbb44e9dd/users/07e98dab752748f39e59562f37384524/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:13.339389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.411202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:13.427861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796be2d-e04f-4d6f-9896-721fb5bf49df admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.431664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.432433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.432721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.433012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.438142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0c694-73d8-4b95-8cb4-a3a56b1cee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.438904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0c694-73d8-4b95-8cb4-a3a56b1cee73 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.439287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0c694-73d8-4b95-8cb4-a3a56b1cee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.439632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0c694-73d8-4b95-8cb4-a3a56b1cee73 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.440457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0c694-73d8-4b95-8cb4-a3a56b1cee73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.442458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0c694-73d8-4b95-8cb4-a3a56b1cee73 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.464472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:13.484667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1007/2014] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:13 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:13.486913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.487281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.487675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.517390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.519923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.529413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] MFA Rules not processed for user `07e98dab752748f39e59562f37384524`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:13.530890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:13.532550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1008/2015] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:13.544346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.566081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd3ff0-7b80-4fa3-86fa-349e43fcaabb service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.586237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.587056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.587508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.587973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.589034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.591859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.612883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1009/2016] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:13.614187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.634573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25bcc-8aaf-406e-84fa-a6d9b0997a12 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.674542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.675423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.675712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.676691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.677590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.679383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1008/2017] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:13.680320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.690204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.705103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027efa6e-af0b-46a6-85c4-5347b289e7f9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.709360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1010/2018] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:13.715400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.715995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.716107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.716353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.716628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.717304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.719223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.786421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1009/2019] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:13 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:13.787428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.802497 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b24964-13c3-46bc-ba18-5444c321f3f6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.874055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.874712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.874994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.875268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.877415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:13.918446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03572b5a-3471-41aa-a453-27bd6f4b8a74 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:13.932058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:13.932913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:13.933304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:13.933663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] PATH_INFO: `/v3/projects/df15313a4f7e4639beeed42fcf5c5d04/users/5a124c625bba4b9585f1340dcc7a88b1/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:13.955305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df15313a4f7e4639beeed42fcf5c5d04, user_id=5a124c625bba4b9585f1340dcc7a88b1, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.958331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.975686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:13.977261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:13.988501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1010/2020] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:13 2022] PUT /identity/v3/projects/df15313a4f7e4639beeed42fcf5c5d04/users/5a124c625bba4b9585f1340dcc7a88b1/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:13.989782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:13.991693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.001157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1011/2021] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:14.002148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.034631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:14.051130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41b91-ef3a-47b6-a26f-1d38bb180b64 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.088208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:14.124868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9394f-fc19-4e71-9b00-addbc8fa6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.125096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9771e0-e407-46ac-a91c-273391caba01 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.125766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9394f-fc19-4e71-9b00-addbc8fa6f70 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.126052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9394f-fc19-4e71-9b00-addbc8fa6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.126466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9394f-fc19-4e71-9b00-addbc8fa6f70 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.127153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9394f-fc19-4e71-9b00-addbc8fa6f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.129243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9394f-fc19-4e71-9b00-addbc8fa6f70 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.130761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.131852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.132328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.132831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] PATH_INFO: `/v3/projects/100ad51e2a5d445db89dbd10a73eedc8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.134438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100ad51e2a5d445db89dbd10a73eedc8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.136976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.158833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1012/2022] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:14.158833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798219f0-92b8-450f-bd78-e5075b6e9aaa None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.158833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798219f0-92b8-450f-bd78-e5075b6e9aaa None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.158833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798219f0-92b8-450f-bd78-e5075b6e9aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.195479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-798219f0-92b8-450f-bd78-e5075b6e9aaa None None] MFA Rules not processed for user `5a124c625bba4b9585f1340dcc7a88b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:14.196740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-798219f0-92b8-450f-bd78-e5075b6e9aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.205221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100ad51e2a5d445db89dbd10a73eedc8', 'request_id': 'req-4e598508-a93a-44a7-bc92-7620180d94f6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:14.211428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100ad51e2a5d445db89dbd10a73eedc8', 'request_id': 'req-4e598508-a93a-44a7-bc92-7620180d94f6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:14.218310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100ad51e2a5d445db89dbd10a73eedc8', 'request_id': 'req-4e598508-a93a-44a7-bc92-7620180d94f6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:14.234413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] The token cache is being invalidate because project 100ad51e2a5d445db89dbd10a73eedc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:14.234992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:14.236311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:14.238602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1011/2023] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:13 2022] DELETE /identity/v3/projects/100ad51e2a5d445db89dbd10a73eedc8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:14.239528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.243075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.299466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1013/2024] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:14.299466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.299466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.299466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.319908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:14.321341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.330796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1014/2025] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:14.338117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.339383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.347561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e598508-a93a-44a7-bc92-7620180d94f6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.355075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4d246-ffbc-49a6-b6af-44ad5aafcef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.355997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d246-ffbc-49a6-b6af-44ad5aafcef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.356125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488db5a6-cadc-4f5a-acd8-1352a5dfc98f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.356339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d246-ffbc-49a6-b6af-44ad5aafcef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.356681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d246-ffbc-49a6-b6af-44ad5aafcef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.357585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d246-ffbc-49a6-b6af-44ad5aafcef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.359447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d246-ffbc-49a6-b6af-44ad5aafcef2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.365599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.365599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.365900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.366131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.366824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.371265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1012/2026] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:14.374431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.376357 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.376652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.376934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.401314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:14.402548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.414156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1013/2027] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:14.415685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.428812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8209356-deeb-483a-b530-62fdcd6ad3ff None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.446532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.447184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.447459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.447797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.449018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.450197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1015/2028] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:14.450679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.450972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.462534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1014/2029] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:14.467449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.477807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c11a-5cc9-4690-a479-828d8a030e4f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.521410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:14.541079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.541720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.542015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.542977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.543692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.545660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.552700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:14.562272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473d41-0dfb-4013-ba6e-6087ab916a60 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.569534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.569731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1015/2030] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:14.570699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.572822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.573396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.574249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.585485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed5a62-6bb4-4e68-be1e-636e526ba2e5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.586897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:14.587690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.590386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.591074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.591184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.591357 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.591635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] PATH_INFO: `/v3/projects/52f123d3aebf450e84d1a187a7f70b56/users/1eb7359bca1f4ee0bc664fddafcbd404/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.612657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f123d3aebf450e84d1a187a7f70b56, user_id=1eb7359bca1f4ee0bc664fddafcbd404, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.615557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1016/2031] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:14.616470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.616811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.630887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499eaea-bc4f-4352-97f9-07b610dfcaa1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.645052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1016/2032] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:14 2022] PUT /identity/v3/projects/52f123d3aebf450e84d1a187a7f70b56/users/1eb7359bca1f4ee0bc664fddafcbd404/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:14.645052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.716778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb42a1f-76fa-4175-848a-e6ad33b6925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.717500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42a1f-76fa-4175-848a-e6ad33b6925e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.717851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42a1f-76fa-4175-848a-e6ad33b6925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.718206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42a1f-76fa-4175-848a-e6ad33b6925e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.718959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb42a1f-76fa-4175-848a-e6ad33b6925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.720673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb42a1f-76fa-4175-848a-e6ad33b6925e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.723103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b322d7-9f54-4a1f-96f6-cb937b3f1e14 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.727703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.728351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.728631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.728901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.729557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.730762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1017/2033] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:14.731975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.732265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.732542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.732640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.763784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1017/2034] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:14.763984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] MFA Rules not processed for user `1eb7359bca1f4ee0bc664fddafcbd404`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:14.764360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:14.764991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.848247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b12b-ba8d-4508-94c9-a382c6540027 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.854596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1018/2035] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:14.854596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.857883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.858897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.858897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.859176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] PATH_INFO: `/v3/projects/ca69d5885106470fa4b4b9726be8a59a/users/ccf97081cb114e61978db259e9cc0852/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.876324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48295c23-3393-45ef-9f91-1afcf7005ca2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.878201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca69d5885106470fa4b4b9726be8a59a, user_id=ccf97081cb114e61978db259e9cc0852, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.881032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.881995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.881995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.882234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.882479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.883196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.885181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.895977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1018/2036] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:14 2022] PUT /identity/v3/projects/ca69d5885106470fa4b4b9726be8a59a/users/ccf97081cb114e61978db259e9cc0852/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:14.897205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.916368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1019/2037] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/users => generated 291 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:14.917603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:14.922283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:14.931187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90be8-6b1a-48dd-a8a6-b9a07ff2a82e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.975640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f0b82-e43f-4ffe-b1d7-959e80dcb0ed admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:14.980494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.981153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.981433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.981715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.982385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:14.984335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:14.990014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:14.997083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:14.997807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:14.998307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:14.998589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:14.999351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.001015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.011826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1019/2038] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:14 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:15.012865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.015567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1020/2039] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:15.016447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.028868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7706b5-3b62-4356-ad59-f8a0626426ef admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.082819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd1e80-2883-4a60-90a8-4e151d1c2750 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.088689 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.089408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.089659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.089884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.090638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.092387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.094140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.094898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.095180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.095453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.096209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.098512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.103563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1020/2040] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:15.107291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.118874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4f459-6152-4e54-9bbd-dae33e151d8c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.122646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1021/2041] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:15 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:15.122646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.126238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.126918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.127204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.127482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] PATH_INFO: `/v3/projects/ca69d5885106470fa4b4b9726be8a59a/users/ccf97081cb114e61978db259e9cc0852/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.129471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca69d5885106470fa4b4b9726be8a59a, user_id=ccf97081cb114e61978db259e9cc0852, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.130451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc817fde-0621-4a8f-a7cb-50da41768b92 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.132202 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.135017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504ed66-3d24-40fb-8914-11a93d8647e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.135792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504ed66-3d24-40fb-8914-11a93d8647e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.136396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504ed66-3d24-40fb-8914-11a93d8647e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.136677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504ed66-3d24-40fb-8914-11a93d8647e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.137364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504ed66-3d24-40fb-8914-11a93d8647e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.138951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504ed66-3d24-40fb-8914-11a93d8647e6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.144486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1021/2042] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:15 2022] PUT /identity/v3/projects/ca69d5885106470fa4b4b9726be8a59a/users/ccf97081cb114e61978db259e9cc0852/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:15.145282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.159819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1022/2043] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:15.159819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.159819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.159819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.188162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] MFA Rules not processed for user `ccf97081cb114e61978db259e9cc0852`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:15.189404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.213078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454d386-fa7d-4cd4-a789-a6f633c48f8e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.218052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.218683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.218966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.219242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.219969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.221575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.233529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1022/2044] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:15.235010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.246500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05222d-a0e4-470c-92ac-f86bb55e503d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.251087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.252400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.252690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.252965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] PATH_INFO: `/v3/projects/068fb368473f40fbab31a938fb3e0e5b/users/560cedf19f6a44169da9280e287a62d0/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.266260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1023/2045] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:15.267095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.273120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=068fb368473f40fbab31a938fb3e0e5b, user_id=560cedf19f6a44169da9280e287a62d0, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.275927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.279199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:15.279324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69261f3f-567a-450f-a066-70d5ef5417d5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.284187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686833b-5438-4c8d-bea2-33351abc7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.284870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686833b-5438-4c8d-bea2-33351abc7db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.285156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686833b-5438-4c8d-bea2-33351abc7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.285416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686833b-5438-4c8d-bea2-33351abc7db7 admin admin] PATH_INFO: `/v3/projects/09dd1cb15a3d46ca8cb22559085f24d1/users/13b43cdd8dea4983b6d1b54ca7d93388/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.292215 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1023/2046] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:15 2022] PUT /identity/v3/projects/068fb368473f40fbab31a938fb3e0e5b/users/560cedf19f6a44169da9280e287a62d0/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:15.292843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.294586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.301436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686833b-5438-4c8d-bea2-33351abc7db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dd1cb15a3d46ca8cb22559085f24d1, user_id=13b43cdd8dea4983b6d1b54ca7d93388, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.304775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686833b-5438-4c8d-bea2-33351abc7db7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.320366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1024/2047] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:15 2022] PUT /identity/v3/projects/09dd1cb15a3d46ca8cb22559085f24d1/users/13b43cdd8dea4983b6d1b54ca7d93388/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:15.321919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.322201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.322627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.354174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541217da-4327-4c35-ba00-d4ea67317185 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.354368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] MFA Rules not processed for user `560cedf19f6a44169da9280e287a62d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:15.355548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.375418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:15.402142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.404317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.404631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.404939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.426796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1025/2048] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:15.426796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.474278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.475278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.485622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1024/2049] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:15.486626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.488024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.498973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017a10-09b6-4d52-9f93-8fa62cf49fa3 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.503089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.503982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.504266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.504405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.505980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed19ae-bdc5-47bf-9040-abe7243f85cf None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.511939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.512605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.512886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.513213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.513917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.514172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.515606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.515718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.525775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1025/2050] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:15.532315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1026/2051] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:15.533404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.546739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.550274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf7f84-d9c9-4311-8245-028a032047d2 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.557260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb719-6aec-4482-8e0e-75c67358ef9f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.557389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118efe91-415a-4065-aace-708a8651686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.558467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118efe91-415a-4065-aace-708a8651686d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.558919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118efe91-415a-4065-aace-708a8651686d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.559560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118efe91-415a-4065-aace-708a8651686d admin admin] PATH_INFO: `/v3/projects/09dd1cb15a3d46ca8cb22559085f24d1/users/13b43cdd8dea4983b6d1b54ca7d93388/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.561821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118efe91-415a-4065-aace-708a8651686d admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dd1cb15a3d46ca8cb22559085f24d1, user_id=13b43cdd8dea4983b6d1b54ca7d93388, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.562356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.563094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.563484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.563903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] PATH_INFO: `/v3/users/560cedf19f6a44169da9280e287a62d0` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.565314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] RBAC: Authorizing `identity:delete_user(user_id=560cedf19f6a44169da9280e287a62d0)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.565429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118efe91-415a-4065-aace-708a8651686d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.579733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.596005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1026/2052] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:15 2022] PUT /identity/v3/projects/09dd1cb15a3d46ca8cb22559085f24d1/users/13b43cdd8dea4983b6d1b54ca7d93388/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:15.598585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.598952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.599290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.634354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560cedf19f6a44169da9280e287a62d0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:15.635313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560cedf19f6a44169da9280e287a62d0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:15.644105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] MFA Rules not processed for user `13b43cdd8dea4983b6d1b54ca7d93388`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:15.645681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.646609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560cedf19f6a44169da9280e287a62d0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:15.647482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560cedf19f6a44169da9280e287a62d0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:15.672414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e6e5beec-541e-4c79-8546-40f92031555e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560cedf19f6a44169da9280e287a62d0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:15.673472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:15.696033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1027/2053] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:15 2022] DELETE /identity/v3/users/560cedf19f6a44169da9280e287a62d0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:15.700490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.700876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.701366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.727298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:15.728646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.758376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1027/2054] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:15.764101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:15.796981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.798646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.820297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1028/2055] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:15.831833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.833511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.877944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f123df-cfee-41a8-a270-ea33e20fc8f0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.896894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912c0bb-1d6b-472d-aa7f-c086528f3507 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.902584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.903621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.903946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.909741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.920496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:15.921961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:15.932109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1029/2056] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:15.935127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:15.935990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:15.936441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:15.936900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:15.939424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.968561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:15.970687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:15.986249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b6c9e-6ff5-4b0e-bfb4-29291d3fafad service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:15.999814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.000806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.000932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.001654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] PATH_INFO: `/v3/users/5a124c625bba4b9585f1340dcc7a88b1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.006770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a124c625bba4b9585f1340dcc7a88b1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.012472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.022025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.023722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e9f8c-71a1-43b3-a36f-43bb543d5a43 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.046510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1028/2057] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:16.081494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a124c625bba4b9585f1340dcc7a88b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.082713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a124c625bba4b9585f1340dcc7a88b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.093755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a124c625bba4b9585f1340dcc7a88b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.094560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a124c625bba4b9585f1340dcc7a88b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.103525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a124c625bba4b9585f1340dcc7a88b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.133860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1030/2058] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:15 2022] DELETE /identity/v3/users/5a124c625bba4b9585f1340dcc7a88b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:16.138969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.138969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.139170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.160439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:16.161801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.192567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:16.263892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1029/2059] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:16.274981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.277036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.330448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.331978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.341880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:16.348152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54e7f-bb26-443c-a343-a0f8f37322bf None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.354711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.355483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.355789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.356073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] PATH_INFO: `/v3/projects/068fb368473f40fbab31a938fb3e0e5b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.357663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=068fb368473f40fbab31a938fb3e0e5b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.358867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499bd8-252c-4b05-b75c-a03f7ac0f77d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.360781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.364472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2532ca7-9721-41fe-9f59-0036879959d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.365222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2532ca7-9721-41fe-9f59-0036879959d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.368214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2532ca7-9721-41fe-9f59-0036879959d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.369406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2532ca7-9721-41fe-9f59-0036879959d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.380571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2532ca7-9721-41fe-9f59-0036879959d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.382071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2532ca7-9721-41fe-9f59-0036879959d9 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.392319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1031/2060] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:16.460573 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '068fb368473f40fbab31a938fb3e0e5b', 'request_id': 'req-5022045a-8d4a-497a-893e-c180c2f92e9e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.470863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '068fb368473f40fbab31a938fb3e0e5b', 'request_id': 'req-5022045a-8d4a-497a-893e-c180c2f92e9e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.477866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '068fb368473f40fbab31a938fb3e0e5b', 'request_id': 'req-5022045a-8d4a-497a-893e-c180c2f92e9e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.492525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] The token cache is being invalidate because project 068fb368473f40fbab31a938fb3e0e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:16.493168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.494020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5022045a-8d4a-497a-893e-c180c2f92e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.495758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1030/2061] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:16 2022] DELETE /identity/v3/projects/068fb368473f40fbab31a938fb3e0e5b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:16.501475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.501834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.502198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.522823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:16.524264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.587880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1032/2062] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:16.595670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.596052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.596361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.617823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:16.618990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.627671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1031/2063] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:16.632132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.642138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b26e8b-e3c7-4ecb-ba33-dc7ece84655f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.646978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.647595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.647907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.648226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.648924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:16.649230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.650835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.663344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1033/2064] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:16.668373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.677729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5db6d-a3ec-4c54-8519-bed65f19f435 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.682921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.683554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.683861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.684130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.685193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.686294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.687106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.688634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.707225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397569e-dfa0-48c0-b33a-457e5976d14d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.718833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.722336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1032/2065] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:16 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:16.722468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.722468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.722468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] PATH_INFO: `/v3/projects/df15313a4f7e4639beeed42fcf5c5d04` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.722468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df15313a4f7e4639beeed42fcf5c5d04)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.723275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.723371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.770876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df15313a4f7e4639beeed42fcf5c5d04', 'request_id': 'req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.776402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df15313a4f7e4639beeed42fcf5c5d04', 'request_id': 'req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.782981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df15313a4f7e4639beeed42fcf5c5d04', 'request_id': 'req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.783641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b403-e7d1-4283-94de-2b30224153b0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.797100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] The token cache is being invalidate because project df15313a4f7e4639beeed42fcf5c5d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:16.797386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.798033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.799671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1034/2066] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:16 2022] DELETE /identity/v3/projects/df15313a4f7e4639beeed42fcf5c5d04 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:16.803791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.805566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.814951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:16.843847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.844481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.844749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.845022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.845702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.847614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.869165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1033/2067] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:16 2022] POST /identity/v3/users => generated 281 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:16.869910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.871453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:16.877193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2bafe-8850-4af8-b728-7457b1b7f8e2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.882344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.882816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77af45d-e455-4736-83f9-5f81feb2d518 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:16.883267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.883348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.883627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] PATH_INFO: `/v3/users/ccf97081cb114e61978db259e9cc0852` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.884987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf97081cb114e61978db259e9cc0852)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.886971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.888220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:16.888727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:16.888846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.889119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.889387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] PATH_INFO: `/v3/users/38283e25ba9d42ed91d8da2d42269bb0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.890988 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38283e25ba9d42ed91d8da2d42269bb0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:16.892941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:16.941140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf97081cb114e61978db259e9cc0852'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.941810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf97081cb114e61978db259e9cc0852'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.943348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38283e25ba9d42ed91d8da2d42269bb0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.944307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38283e25ba9d42ed91d8da2d42269bb0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.949351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf97081cb114e61978db259e9cc0852'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.950032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf97081cb114e61978db259e9cc0852'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.951478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38283e25ba9d42ed91d8da2d42269bb0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.952120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38283e25ba9d42ed91d8da2d42269bb0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.958526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf97081cb114e61978db259e9cc0852'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.961560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-f00a26ca-c5e5-45d5-b7c3-f8258c7a1773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38283e25ba9d42ed91d8da2d42269bb0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:16.977699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1035/2068] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:16 2022] DELETE /identity/v3/users/ccf97081cb114e61978db259e9cc0852 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:16.980549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1034/2069] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:16 2022] DELETE /identity/v3/users/38283e25ba9d42ed91d8da2d42269bb0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:16.984687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:16.985025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:16.985497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:16.986081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:16.986573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.025588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:17.027162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.067373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9cc79-8903-48b2-a39e-003fe2719082 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.129373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1035/2070] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:17.141666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.143482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.144159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.145143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.145394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.145720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.146772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.148784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.160721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1036/2071] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:17.164445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.166377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.180523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464fcd2-07b9-41a7-b3bb-1de742e2bb4d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.186433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.186433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.186433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.186433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.192942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.195366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.204119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:17.215381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1037/2072] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:17.216724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.231035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb783f-7906-4f2b-956b-76da60181abc None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.231567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd661739-ab86-479c-8d2b-8088c8666ab3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.235620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e1f76-fb6c-4331-b657-8a3bde4f762e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.236288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e1f76-fb6c-4331-b657-8a3bde4f762e service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.236568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e1f76-fb6c-4331-b657-8a3bde4f762e service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.236843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e1f76-fb6c-4331-b657-8a3bde4f762e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.250588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.250588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.250588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.250588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] PATH_INFO: `/v3/projects/1e747d5446624cfeaca8006f7637b29c/users/08a3ccab594740388ca4d26b9887d04e/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.253515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:17.259373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e1f76-fb6c-4331-b657-8a3bde4f762e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.261223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e1f76-fb6c-4331-b657-8a3bde4f762e service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.266006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e747d5446624cfeaca8006f7637b29c, user_id=08a3ccab594740388ca4d26b9887d04e, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.269332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.271608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1036/2073] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:17.286273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1038/2074] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:17 2022] PUT /identity/v3/projects/1e747d5446624cfeaca8006f7637b29c/users/08a3ccab594740388ca4d26b9887d04e/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:17.299799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.300281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.300608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.332247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] MFA Rules not processed for user `08a3ccab594740388ca4d26b9887d04e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:17.333464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.347087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.349059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.426347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1037/2075] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:17.463118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.464717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.471893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e65e-ba6a-4485-9f66-30433ba7c66b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.479360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.480778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.481063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.481386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] PATH_INFO: `/v3/projects/ca69d5885106470fa4b4b9726be8a59a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.482859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca69d5885106470fa4b4b9726be8a59a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.485114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.563564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646187-38ec-4a05-84a3-9e97be6512eb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.574859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca69d5885106470fa4b4b9726be8a59a', 'request_id': 'req-be5d69a2-a088-4528-b97a-30d28dcaec69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.584556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca69d5885106470fa4b4b9726be8a59a', 'request_id': 'req-be5d69a2-a088-4528-b97a-30d28dcaec69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.597465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca69d5885106470fa4b4b9726be8a59a', 'request_id': 'req-be5d69a2-a088-4528-b97a-30d28dcaec69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.624254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] The token cache is being invalidate because project ca69d5885106470fa4b4b9726be8a59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:17.624551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.626383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-be5d69a2-a088-4528-b97a-30d28dcaec69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.628246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1039/2076] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:17 2022] DELETE /identity/v3/projects/ca69d5885106470fa4b4b9726be8a59a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:17.632436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.633429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.633713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.634006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.635935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.636217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.636490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.638260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.657204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:17.658467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.712678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:17.734599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1040/2077] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:17.735520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.736920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.742854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.744346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.750505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afbe8d-3610-425d-9b20-c80b70d6fdd1 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.753509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1038/2078] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:17.754361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.755627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.755892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.756573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.756851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.757259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] PATH_INFO: `/v3/projects/fa4ebdb7cabf464cb9f6bddddb37cdc5` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.758835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4ebdb7cabf464cb9f6bddddb37cdc5)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.769506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d9a1-2447-483a-9450-61969dd7f29b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.769844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.773764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.774407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.774688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.774958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.777608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.786218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:17.825175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4ebdb7cabf464cb9f6bddddb37cdc5', 'request_id': 'req-7842a607-575b-482f-babb-b1cc4bbf349a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.830748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4ebdb7cabf464cb9f6bddddb37cdc5', 'request_id': 'req-7842a607-575b-482f-babb-b1cc4bbf349a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.837334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4ebdb7cabf464cb9f6bddddb37cdc5', 'request_id': 'req-7842a607-575b-482f-babb-b1cc4bbf349a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.843086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.844850 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.850456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] The token cache is being invalidate because project fa4ebdb7cabf464cb9f6bddddb37cdc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:17.850744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.851414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7842a607-575b-482f-babb-b1cc4bbf349a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.853405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1041/2079] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:17 2022] DELETE /identity/v3/projects/fa4ebdb7cabf464cb9f6bddddb37cdc5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:17.856532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1039/2080] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:17.856532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.856697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.860562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.860674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.860906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.876882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221085f6-29ad-4dff-8e0a-f09418c642f5 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.882362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:17.883532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.941879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.942565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.942845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.943121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] PATH_INFO: `/v3/users/08a3ccab594740388ca4d26b9887d04e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.944482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a3ccab594740388ca4d26b9887d04e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.946443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.950704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1042/2081] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:17.951625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:17.952881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:17.965241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd973611-e567-48c1-a4f0-5b99967797c4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:17.971733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:17.972447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:17.972731 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:17.973398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] PATH_INFO: `/v3/users/3fabcc997db245c39f9c586250905df1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:17.974582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fabcc997db245c39f9c586250905df1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:17.976693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:17.999002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a3ccab594740388ca4d26b9887d04e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:17.999692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a3ccab594740388ca4d26b9887d04e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.008169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a3ccab594740388ca4d26b9887d04e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.009379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a3ccab594740388ca4d26b9887d04e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.018185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a3ccab594740388ca4d26b9887d04e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.033333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fabcc997db245c39f9c586250905df1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.034066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:18.034203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fabcc997db245c39f9c586250905df1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.043871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fabcc997db245c39f9c586250905df1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.044728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1040/2082] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:17 2022] DELETE /identity/v3/users/08a3ccab594740388ca4d26b9887d04e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:18.044849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fabcc997db245c39f9c586250905df1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.048674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.050149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.055674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-83b673e7-d6b7-4d9a-b497-cab4609ef0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fabcc997db245c39f9c586250905df1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.075700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1043/2083] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:17 2022] DELETE /identity/v3/users/3fabcc997db245c39f9c586250905df1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:18.077998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.078391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.084331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.112087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:18.113571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.124416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:18.130012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c005f4-d5a5-4f12-8f26-44f1e48608ae admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.199805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1044/2084] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:18.207264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.208110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.208395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.208659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.209390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:18.209694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.210013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.211359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.211472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.227235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1041/2085] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:18.231895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.233684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.247627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd2b76-1d1e-4b78-a499-afa0e7635ffb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.274382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.275348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.275834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.276288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.277164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.280608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.308475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df421f7f-76df-4521-9307-60b507af8f86 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.313404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.314343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.315191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.315644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.328437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1042/2086] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:18 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:18.330623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.332977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.335985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.353079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1045/2087] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:18.404546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ec2d0-a5cb-4d51-9c28-abf3cea98394 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.410238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.410869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.411331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.411452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.412121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.415833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.442471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1043/2088] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:18 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:18.446503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.460581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ccf46-a78a-42e8-ac02-0c3b8bec9fba service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.466185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.466893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.467225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.467553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.468300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.470097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.482564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1046/2089] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:18.484725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.487141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.492732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.499064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1893126-5d5d-4504-aced-7d2e99b54602 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.505544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.506414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.506821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.506963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.507907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.510394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.510820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:18.514642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75563e0-6956-46a1-881b-4c6689293919 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.521384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.522150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.522704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1047/2090] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:18.526209 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.527473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] PATH_INFO: `/v3/projects/b4e045b431214806a0745389172ff308` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.528604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.529387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e045b431214806a0745389172ff308)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.533145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.542181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d587d-ab03-4445-a7f3-826227aed42d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.548604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.549996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.550147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.550471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] PATH_INFO: `/v3/projects/71fa01660bbf481c810ecb6374e4bdbf/users/14ea77c59f54488ea9fe28bb821b5dea/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.570309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fa01660bbf481c810ecb6374e4bdbf, user_id=14ea77c59f54488ea9fe28bb821b5dea, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.574728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.604453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1048/2091] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:18 2022] PUT /identity/v3/projects/71fa01660bbf481c810ecb6374e4bdbf/users/14ea77c59f54488ea9fe28bb821b5dea/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:18.605916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.608047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.645354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e045b431214806a0745389172ff308', 'request_id': 'req-52710987-0151-41ad-a10c-d8c5976a51a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.651927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e045b431214806a0745389172ff308', 'request_id': 'req-52710987-0151-41ad-a10c-d8c5976a51a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.662191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e045b431214806a0745389172ff308', 'request_id': 'req-52710987-0151-41ad-a10c-d8c5976a51a4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.680781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] The token cache is being invalidate because project b4e045b431214806a0745389172ff308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:18.681059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.681671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-52710987-0151-41ad-a10c-d8c5976a51a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.683472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1044/2092] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:18 2022] DELETE /identity/v3/projects/b4e045b431214806a0745389172ff308 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:18.684459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26df51-3531-4fb3-be91-6725ea400dac None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.684737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26df51-3531-4fb3-be91-6725ea400dac None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.685007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26df51-3531-4fb3-be91-6725ea400dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.708026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f14d-3ae3-47e8-982c-f724770102ea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.710127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fd26df51-3531-4fb3-be91-6725ea400dac None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:18.713677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fd26df51-3531-4fb3-be91-6725ea400dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.751059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:18.776276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.776909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.777323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.777600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.779437 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.785008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1045/2093] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:18.786336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.786455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.786804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.809304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:18.810619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.827105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1046/2094] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:18.827105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.827105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.866313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.867940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.877904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1049/2095] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:18.878679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:18.880107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.888643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d91e44-4086-49fa-a08d-171850b0c243 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.892873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12477f8-f60a-49f2-95ca-bda8e7304515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.893500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12477f8-f60a-49f2-95ca-bda8e7304515 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.893779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12477f8-f60a-49f2-95ca-bda8e7304515 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.894054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bade75-2b12-4b5e-a180-948bb47bf5de service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:18.894159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12477f8-f60a-49f2-95ca-bda8e7304515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.898610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:18.899240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.899518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.899816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] PATH_INFO: `/v3/projects/1e747d5446624cfeaca8006f7637b29c` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.900979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e747d5446624cfeaca8006f7637b29c)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.902920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.903979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12477f8-f60a-49f2-95ca-bda8e7304515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:18.905780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12477f8-f60a-49f2-95ca-bda8e7304515 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:18.920335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1047/2096] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:18.921648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:18.921889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:18.922157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:18.944752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:18.946138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:18.965311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e747d5446624cfeaca8006f7637b29c', 'request_id': 'req-99e4465e-3458-41c6-91c6-d3145ec02b48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.976762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e747d5446624cfeaca8006f7637b29c', 'request_id': 'req-99e4465e-3458-41c6-91c6-d3145ec02b48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:18.982990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:18.989373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e747d5446624cfeaca8006f7637b29c', 'request_id': 'req-99e4465e-3458-41c6-91c6-d3145ec02b48'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:19.017688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] The token cache is being invalidate because project 1e747d5446624cfeaca8006f7637b29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:19.017989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:19.018805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-99e4465e-3458-41c6-91c6-d3145ec02b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:19.026706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1050/2097] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:18 2022] DELETE /identity/v3/projects/1e747d5446624cfeaca8006f7637b29c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:19.030387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d3d2e-1fed-492f-a832-f1a3f0046c11 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.030387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d3d2e-1fed-492f-a832-f1a3f0046c11 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.030387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d3d2e-1fed-492f-a832-f1a3f0046c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.059534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1048/2098] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:19.064148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.070138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-741d3d2e-1fed-492f-a832-f1a3f0046c11 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:19.071479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-741d3d2e-1fed-492f-a832-f1a3f0046c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.074299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e1fe-f5e8-4eb9-966d-97041efc9630 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.079976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.080922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.081486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.081606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1051/2099] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:19.082370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.083199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:19.083511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.085244 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.090970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.091576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.092082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.098798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1049/2100] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:14:19.104594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.115335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:19.115965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44342d-3949-4bf7-bbfc-ec70ab7a710c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.116964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.122873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.123394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.123559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.124037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.124632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.126662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1052/2101] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:19.128536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.132012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.141473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9051-5b61-47a8-a666-29d28ad44fc0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.147078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.147806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.148183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.148528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.149325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:19.149794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.151533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.162508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1050/2102] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:19.167292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.172414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1053/2103] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:19.173768 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.185435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be18b-19f1-49ff-a123-d943dbe21037 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.229462 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:19.238248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b732e7e-3c77-44c5-8ee7-201c2d203076 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.244694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.245556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.245833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.246349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.247152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.249199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.268537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.269238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.269519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.269786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.270481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.272416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.276887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1051/2104] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:19.280863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.291585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a0dc-acd8-4d4d-bf89-70b6eb15dd98 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.297346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.298038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.298384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.298685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.299793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.301297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.313655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1054/2105] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:19.314710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.327181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1052/2106] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:19.331721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.345912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64def5a1-7a22-4e78-9057-26266dc3bbfb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.389619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70376334-febe-41fd-8f6a-45f362bf84d4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.396492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.397564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.398265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.398547 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.399398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.402532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.450657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1055/2107] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:19.451886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.452678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.452831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.453107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.453801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.454942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.455475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.465329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e1d8f-e0cd-46c4-925a-9a985e9ed628 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.466512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1053/2108] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:19.470243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.470920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.471080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.471351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.471630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.472527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.481559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.481996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:19.490987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591a211-ce1e-4a73-b8d3-968bca25858e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.495843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1056/2109] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:19.500606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81480c7e-4e1f-4969-bfbc-2256da66e8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.501223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.501335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81480c7e-4e1f-4969-bfbc-2256da66e8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.501595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81480c7e-4e1f-4969-bfbc-2256da66e8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.502738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81480c7e-4e1f-4969-bfbc-2256da66e8f0 admin admin] PATH_INFO: `/v3/projects/8a2b6b65ed3d40e7aaf0bd00b458533e/users/bb9494123ca1454ebac70ae38cbc70c7/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.514530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95f50-970e-4791-a77b-6e9869da93d3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.524225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.525173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.525609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.526097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.527481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.529901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.540257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81480c7e-4e1f-4969-bfbc-2256da66e8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a2b6b65ed3d40e7aaf0bd00b458533e, user_id=bb9494123ca1454ebac70ae38cbc70c7, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.543429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81480c7e-4e1f-4969-bfbc-2256da66e8f0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.547429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1057/2110] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:19.548671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.562773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae002-5a5d-41ae-bb42-9e2944053a12 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.564002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1054/2111] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:19 2022] PUT /identity/v3/projects/8a2b6b65ed3d40e7aaf0bd00b458533e/users/bb9494123ca1454ebac70ae38cbc70c7/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:19.575474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca91c-464c-481a-8f2e-aebf87b4254f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.575757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca91c-464c-481a-8f2e-aebf87b4254f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.576098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca91c-464c-481a-8f2e-aebf87b4254f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.614677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4b4ca91c-464c-481a-8f2e-aebf87b4254f None None] MFA Rules not processed for user `bb9494123ca1454ebac70ae38cbc70c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:19.615649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4b4ca91c-464c-481a-8f2e-aebf87b4254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.644766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c25e0-6dd4-43ff-bd68-ccc65c2ed3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.645794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c25e0-6dd4-43ff-bd68-ccc65c2ed3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.646088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c25e0-6dd4-43ff-bd68-ccc65c2ed3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.646356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c25e0-6dd4-43ff-bd68-ccc65c2ed3a8 admin admin] PATH_INFO: `/v3/projects/37a44af0e30841049a0c40b42790dba8/users/527be219ee9e44c18f2e0f24e129a76a/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.664433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c25e0-6dd4-43ff-bd68-ccc65c2ed3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a44af0e30841049a0c40b42790dba8, user_id=527be219ee9e44c18f2e0f24e129a76a, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.668565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c25e0-6dd4-43ff-bd68-ccc65c2ed3a8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.699160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1058/2112] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:19 2022] PUT /identity/v3/projects/37a44af0e30841049a0c40b42790dba8/users/527be219ee9e44c18f2e0f24e129a76a/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:19.710017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.710290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.710629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.736824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1055/2113] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:19.736824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.736824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.736824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.739568 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:19.753158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] MFA Rules not processed for user `527be219ee9e44c18f2e0f24e129a76a`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:19.757204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.770149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] MFA Rules not processed for user `14ea77c59f54488ea9fe28bb821b5dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:19.771375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.848986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1059/2114] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:19.849939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.879385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1056/2115] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:19.880696 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.882060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.920515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb404e5-70db-4ef6-ac5e-dfce21cd4171 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:19.926607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:19.927367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:19.927749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:19.928188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:19.929212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:19.931326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:19.964053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1060/2116] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:19.964974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:19.966589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:19.972952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:19.986856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12fb59-484a-4ee5-9bda-e422cab158f6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.035294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.036100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.036377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.036658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.041442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b30ec9-96b2-4e00-b06b-e39fa609c51b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.046054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.046681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.046959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.047239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.047944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.049862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.079980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1061/2117] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:19 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:20.080690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.092770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:20.107347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.109081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.119239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1057/2118] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:20.120324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.132299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c6ada-913a-4171-af4a-5a00fc5eb98a service glance] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.150178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8356f1-6a1c-4bff-aba0-4eb41e6943ea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.159190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.159865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.160154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.160433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.161648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.163865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.196532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1062/2119] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:20 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:20.201258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.213140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.213873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.214212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.214560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.215302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.217525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.218451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c065c-dcd2-49a7-ac8f-51d47549d950 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.227864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.229814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.230346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.230705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.231892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.233826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.246532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1058/2120] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:20 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:20.249827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.251191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1063/2121] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:20.255576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.263471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e439d0-00aa-4207-b019-c4eb20f95545 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.269078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.269479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5b603-540b-4286-b4ca-20039b09b24e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.269837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.270185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.270529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.271360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.273528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.276015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.276785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.277147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.277519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.278263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.280042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.290281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1059/2122] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:20.291177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.295210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1064/2123] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:20.300653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.301508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe85bf-c313-4944-96e9-f312b3b83a4c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.307235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.308039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.308331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.308623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] PATH_INFO: `/v3/projects/a9961b4b8ece496992424ee268a4d57a/users/cfc240246f624dbca3f006f423865ca9/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.313861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f0b6c-a4f8-4431-89e7-7a1a53d16172 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.319452 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0211fc-f255-42c4-9c64-d5f60643a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.321265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0211fc-f255-42c4-9c64-d5f60643a2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.321667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0211fc-f255-42c4-9c64-d5f60643a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.322123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0211fc-f255-42c4-9c64-d5f60643a2b5 admin admin] PATH_INFO: `/v3/projects/da0b85f5e518425680001d378b668aa9/users/77f46a0bd8104ec38b2887931b5e224b/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.326859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9961b4b8ece496992424ee268a4d57a, user_id=cfc240246f624dbca3f006f423865ca9, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.330142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.340486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0211fc-f255-42c4-9c64-d5f60643a2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0b85f5e518425680001d378b668aa9, user_id=77f46a0bd8104ec38b2887931b5e224b, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.343483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0211fc-f255-42c4-9c64-d5f60643a2b5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.347532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1060/2124] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:20 2022] PUT /identity/v3/projects/a9961b4b8ece496992424ee268a4d57a/users/cfc240246f624dbca3f006f423865ca9/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:20.351999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.357861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:20.360820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1065/2125] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:20 2022] PUT /identity/v3/projects/da0b85f5e518425680001d378b668aa9/users/77f46a0bd8104ec38b2887931b5e224b/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:20.371471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.371976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.372582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.436850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] MFA Rules not processed for user `77f46a0bd8104ec38b2887931b5e224b`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:20.438078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:20.441921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38994-a9c7-49ec-9d34-f10e43085ad4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.529908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.530965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.531428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.532051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.533332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.537555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.556644 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1066/2126] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:20.557941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.563254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1061/2127] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:20.564845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.579427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ca8c-9ff6-4beb-8cf1-b5c95e577ca5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.584236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1003054-61a9-4d29-bca2-187827abe1cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.586934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.587738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.588298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.588730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] PATH_INFO: `/v3/projects/a9961b4b8ece496992424ee268a4d57a/users/cfc240246f624dbca3f006f423865ca9/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.590595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.591585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.591693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9961b4b8ece496992424ee268a4d57a, user_id=cfc240246f624dbca3f006f423865ca9, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.592582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.592931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.593933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.594483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.595996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.612479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1062/2128] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:20 2022] PUT /identity/v3/projects/a9961b4b8ece496992424ee268a4d57a/users/cfc240246f624dbca3f006f423865ca9/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:20.639804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1067/2129] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:20 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:20.643865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.754425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a222ae3-91e4-43de-8752-524aff74c1cc admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.852227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:20.882360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.883647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.883647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.884161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.885103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.887742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.915844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1063/2130] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:20 2022] POST /identity/v3/users => generated 295 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:20.918982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.931319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd751f-bb96-499a-80e8-097e3c2af020 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.936296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.936919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.937197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.937474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.938841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.940645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:20.951019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1068/2131] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:20.955092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:20.966389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc69e-7dc7-4c82-80f6-b142fb160f3e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:20.971366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:20.972221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:20.972505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:20.972847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] PATH_INFO: `/v3/projects/eef6c9a64571442ca4e64215ea1a8f1e/users/dddaf956404c4df0892aeb03c1c221a6/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:20.988724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef6c9a64571442ca4e64215ea1a8f1e, user_id=dddaf956404c4df0892aeb03c1c221a6, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:20.991544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:21.008147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1064/2132] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:20 2022] PUT /identity/v3/projects/eef6c9a64571442ca4e64215ea1a8f1e/users/dddaf956404c4df0892aeb03c1c221a6/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:21.016220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:21.024704 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e93e9-9492-4199-9676-7c7a3a873a7c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:21.029793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:21.088450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72635bef-be74-49a7-951c-9e8f3b17091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:21.088815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72635bef-be74-49a7-951c-9e8f3b17091e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.089161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72635bef-be74-49a7-951c-9e8f3b17091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.090016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72635bef-be74-49a7-951c-9e8f3b17091e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.090265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72635bef-be74-49a7-951c-9e8f3b17091e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:21.092669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72635bef-be74-49a7-951c-9e8f3b17091e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:21.105028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1069/2133] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:21.109268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:21.124031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb589f90-1882-4d52-a6f4-2dd855b461fe admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:21.129920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:21.130669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.131012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.131346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] PATH_INFO: `/v3/projects/eef6c9a64571442ca4e64215ea1a8f1e/users/dddaf956404c4df0892aeb03c1c221a6/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.134341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef6c9a64571442ca4e64215ea1a8f1e, user_id=dddaf956404c4df0892aeb03c1c221a6, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:21.137635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:21.164416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1065/2134] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:21 2022] PUT /identity/v3/projects/eef6c9a64571442ca4e64215ea1a8f1e/users/dddaf956404c4df0892aeb03c1c221a6/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:21.175757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce609891-5e7b-4c2c-8f86-f797c60cfdcb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.176247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce609891-5e7b-4c2c-8f86-f797c60cfdcb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.176591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce609891-5e7b-4c2c-8f86-f797c60cfdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.225283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ce609891-5e7b-4c2c-8f86-f797c60cfdcb None None] MFA Rules not processed for user `dddaf956404c4df0892aeb03c1c221a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:21.227382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ce609891-5e7b-4c2c-8f86-f797c60cfdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:21.329138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1070/2135] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:21.341319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:21.343529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:21.406899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f5ec0-e6d9-4c2c-9469-07e4d5c2cc77 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:21.411310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:21.412012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.412299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.412731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.423486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:21.425768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:21.438245 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1066/2136] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:21.486106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.486695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.487277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.509088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:21.520330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374335f7-9d31-414b-995b-316e6784e1ce service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:21.528712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:21.529806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.530376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.532094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.535292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] MFA Rules not processed for user `cfc240246f624dbca3f006f423865ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:21.536593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:21.629174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:21.630703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:21.641064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1067/2137] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:21.661963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1071/2138] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:21.676949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:21.678109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:21.692040 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce08bdd-da0b-4cf3-9588-958745ac8c2b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:21.697505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:21.698331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:21.698625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:21.698909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:21.752551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:21.777532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:21.778989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:21.791274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1068/2139] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:23.267495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:23.278358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d042-b7cd-49b7-886f-421a4c405d48 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:23.282187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1e42f-1ab7-4406-a7ef-6af2e903d6c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:23.282961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e42f-1ab7-4406-a7ef-6af2e903d6c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:23.283245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e42f-1ab7-4406-a7ef-6af2e903d6c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:23.283663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e42f-1ab7-4406-a7ef-6af2e903d6c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:23.294676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1e42f-1ab7-4406-a7ef-6af2e903d6c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:23.296617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1e42f-1ab7-4406-a7ef-6af2e903d6c5 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:23.306691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1072/2140] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:23.430208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:23.431832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:23.532482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f056700-9993-4785-bd7e-a837877e4679 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:23.556980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:23.557524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:23.557807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:23.558085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] PATH_INFO: `/v3/users/07e98dab752748f39e59562f37384524` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:23.559331 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e98dab752748f39e59562f37384524)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:23.561693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:23.646765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e98dab752748f39e59562f37384524'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:23.654291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e98dab752748f39e59562f37384524'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:23.663693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e98dab752748f39e59562f37384524'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:23.665265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e98dab752748f39e59562f37384524'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:23.680640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e98dab752748f39e59562f37384524'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:23.713510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1069/2141] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:23 2022] DELETE /identity/v3/users/07e98dab752748f39e59562f37384524 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:23.717336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:23.717633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:23.719952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:23.731863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:23.752815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:23.758496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:23.841336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1073/2142] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:23.853471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:23.854928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:23.913249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8aa1d2-8407-4041-8604-591453b8b1ee admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:23.918074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:23.918810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:23.919097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:23.919372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:23.926779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:23.929959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:23.932229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:23.944575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1070/2143] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:24.434840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:24.440696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:24.459978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b853d-6d5f-442f-b70b-b32dd21352b7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:24.465809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:24.466485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:24.466767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:24.467042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] PATH_INFO: `/v3/projects/45fbc498a16b4e5e94f7e59bbb44e9dd` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:24.468246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fbc498a16b4e5e94f7e59bbb44e9dd)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:24.470168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:24.518935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fbc498a16b4e5e94f7e59bbb44e9dd', 'request_id': 'req-2b0bdb3f-bf27-458d-b4af-ee54786ff272'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.524922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fbc498a16b4e5e94f7e59bbb44e9dd', 'request_id': 'req-2b0bdb3f-bf27-458d-b4af-ee54786ff272'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.531540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fbc498a16b4e5e94f7e59bbb44e9dd', 'request_id': 'req-2b0bdb3f-bf27-458d-b4af-ee54786ff272'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.544782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] The token cache is being invalidate because project 45fbc498a16b4e5e94f7e59bbb44e9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:24.546895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.546895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.547184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1074/2144] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:24 2022] DELETE /identity/v3/projects/45fbc498a16b4e5e94f7e59bbb44e9dd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:24.551225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:24.562863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9477ea1-691e-4f3a-9158-5d75aef50279 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:24.631669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:24.632402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:24.632640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:24.633014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] PATH_INFO: `/v3/users/1eb7359bca1f4ee0bc664fddafcbd404` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:24.634208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb7359bca1f4ee0bc664fddafcbd404)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:24.636168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:24.693200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb7359bca1f4ee0bc664fddafcbd404'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.693947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb7359bca1f4ee0bc664fddafcbd404'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.702017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb7359bca1f4ee0bc664fddafcbd404'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.705961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb7359bca1f4ee0bc664fddafcbd404'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.719042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb7359bca1f4ee0bc664fddafcbd404'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:24.742466 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1071/2145] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:24 2022] DELETE /identity/v3/users/1eb7359bca1f4ee0bc664fddafcbd404 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:24.931190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:24.932759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:24.949718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bdb3f-bf27-458d-b4af-ee54786ff272 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:25.090556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:25.091357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.091646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.091957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] PATH_INFO: `/v3/users/bb9494123ca1454ebac70ae38cbc70c7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.093426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9494123ca1454ebac70ae38cbc70c7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:25.095663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:25.138927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:25.160228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9494123ca1454ebac70ae38cbc70c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.160998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9494123ca1454ebac70ae38cbc70c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.168539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9494123ca1454ebac70ae38cbc70c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.169246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9494123ca1454ebac70ae38cbc70c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.176131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3cc8e4c9-4ef4-46a9-94e3-c84345a2ef1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9494123ca1454ebac70ae38cbc70c7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.181160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:25.182669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.198896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1075/2146] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:24 2022] DELETE /identity/v3/users/bb9494123ca1454ebac70ae38cbc70c7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:25.202404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1b01-3c60-43b3-9abc-719cba18f5b7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:25.203440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.203740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.204036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.229332 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:25.230563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.262594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:25.263272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.263556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.263863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] PATH_INFO: `/v3/projects/52f123d3aebf450e84d1a187a7f70b56` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.265171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f123d3aebf450e84d1a187a7f70b56)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:25.266924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:25.286919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:25.297264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1076/2147] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:25.307488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:25.310432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.334096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f123d3aebf450e84d1a187a7f70b56', 'request_id': 'req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.342165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f123d3aebf450e84d1a187a7f70b56', 'request_id': 'req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.349871 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f123d3aebf450e84d1a187a7f70b56', 'request_id': 'req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.366380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] The token cache is being invalidate because project 52f123d3aebf450e84d1a187a7f70b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:25.367123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.368016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.369792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1072/2148] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:25 2022] DELETE /identity/v3/projects/52f123d3aebf450e84d1a187a7f70b56 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:25.373593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:25.374950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.388110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f483b-a237-4f14-bf01-da9daab7defe None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:25.442892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6d4db-ec04-4a7f-87e4-b34f95c53a6c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:25.445912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:25.446622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.446908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.447187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.449339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.449913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:25.450745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.451197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.451617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] PATH_INFO: `/v3/users/13b43cdd8dea4983b6d1b54ca7d93388` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.453589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b43cdd8dea4983b6d1b54ca7d93388)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:25.464154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:25.464406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:25.465498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:25.480671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1077/2149] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:25.524831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b43cdd8dea4983b6d1b54ca7d93388'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.525510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b43cdd8dea4983b6d1b54ca7d93388'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.534186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b43cdd8dea4983b6d1b54ca7d93388'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.534910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b43cdd8dea4983b6d1b54ca7d93388'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.545634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b43cdd8dea4983b6d1b54ca7d93388'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.564125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1073/2150] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:25 2022] DELETE /identity/v3/users/13b43cdd8dea4983b6d1b54ca7d93388 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:25.829358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:25.830745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.842249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:25.846252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ecf93-68e1-49d4-95d7-4cf51c448c0a service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:25.913423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:25.914265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.914609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.915174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] PATH_INFO: `/v3/projects/8a2b6b65ed3d40e7aaf0bd00b458533e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.916403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a2b6b65ed3d40e7aaf0bd00b458533e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:25.918421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:25.926308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:25.932207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:25.946318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e2b5-1fd9-4553-a5b3-ed3865bf05be admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:25.952226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:25.952986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:25.953273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:25.953546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] PATH_INFO: `/v3/projects/09dd1cb15a3d46ca8cb22559085f24d1` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:25.954907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=09dd1cb15a3d46ca8cb22559085f24d1)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:25.957132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:25.974219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a2b6b65ed3d40e7aaf0bd00b458533e', 'request_id': 'req-22cad10e-f5b0-420a-bace-f050a451f35e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.980202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a2b6b65ed3d40e7aaf0bd00b458533e', 'request_id': 'req-22cad10e-f5b0-420a-bace-f050a451f35e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:25.982444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:25.986590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a2b6b65ed3d40e7aaf0bd00b458533e', 'request_id': 'req-22cad10e-f5b0-420a-bace-f050a451f35e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.000088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] The token cache is being invalidate because project 8a2b6b65ed3d40e7aaf0bd00b458533e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:26.000380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.001013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.002543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1078/2151] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:25 2022] DELETE /identity/v3/projects/8a2b6b65ed3d40e7aaf0bd00b458533e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:26.006513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.007899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.013118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09dd1cb15a3d46ca8cb22559085f24d1', 'request_id': 'req-3a67aee6-3d71-4039-ab77-bda2f0d320ec'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.019054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09dd1cb15a3d46ca8cb22559085f24d1', 'request_id': 'req-3a67aee6-3d71-4039-ab77-bda2f0d320ec'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.027261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09dd1cb15a3d46ca8cb22559085f24d1', 'request_id': 'req-3a67aee6-3d71-4039-ab77-bda2f0d320ec'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.041610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] The token cache is being invalidate because project 09dd1cb15a3d46ca8cb22559085f24d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:26.041906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.042530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-3a67aee6-3d71-4039-ab77-bda2f0d320ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.045445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1074/2152] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:25 2022] DELETE /identity/v3/projects/09dd1cb15a3d46ca8cb22559085f24d1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:26.049099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041aac-acb9-47af-9f80-def8ba6ef0a8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.049386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041aac-acb9-47af-9f80-def8ba6ef0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.049661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041aac-acb9-47af-9f80-def8ba6ef0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.079519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ff041aac-acb9-47af-9f80-def8ba6ef0a8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:26.080777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ff041aac-acb9-47af-9f80-def8ba6ef0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.110023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cad10e-f5b0-420a-bace-f050a451f35e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.172903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1075/2153] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:26.181193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.181520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.181903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.192472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:26.225547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:26.227238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.229956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.230993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.231601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.232083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] PATH_INFO: `/v3/users/77f46a0bd8104ec38b2887931b5e224b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.234137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] RBAC: Authorizing `identity:delete_user(user_id=77f46a0bd8104ec38b2887931b5e224b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.236944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.242315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1076/2154] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:26.243889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.255051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601ae2-eaa4-438f-a1dd-97eaf4e41df4 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.263861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.264621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.264907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.265223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.266549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:26.267103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.268875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.293928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1077/2155] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:26.301022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.329875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb7e3-f8db-4af6-a6b1-da29c52734d1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.335450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77f46a0bd8104ec38b2887931b5e224b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.338559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77f46a0bd8104ec38b2887931b5e224b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.342497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.345092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.347411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.347661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77f46a0bd8104ec38b2887931b5e224b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.348097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.348706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77f46a0bd8104ec38b2887931b5e224b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.348897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.353449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.361054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77f46a0bd8104ec38b2887931b5e224b'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.406087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1079/2156] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:26 2022] DELETE /identity/v3/users/77f46a0bd8104ec38b2887931b5e224b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:26.432829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1078/2157] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:26 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:26.437979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.439603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.524947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753995-235d-498c-bc2c-618cc12c499b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.597588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.598290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.599032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.599032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.599711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.601749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.622284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:26.629404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1080/2158] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:26 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:26.633579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.644598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11cf5-5028-4b80-a9c8-badb8bb7cc46 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.649315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.649975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.650253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.650520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.651213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.652919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.666786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1079/2159] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:26.668037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.677943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95ba61-5a8d-40f1-bdc4-e15e283d5c14 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.682769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.683552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.683937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.684279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.685046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.686825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.699394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1081/2160] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:26.704022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.715510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.717235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f351-d99b-486b-a38a-8f3fb51b1f90 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.717369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.721976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2b5ad-2eef-4fde-b296-2dc7c2206120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.723236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2b5ad-2eef-4fde-b296-2dc7c2206120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.723519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2b5ad-2eef-4fde-b296-2dc7c2206120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.723994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2b5ad-2eef-4fde-b296-2dc7c2206120 admin admin] PATH_INFO: `/v3/projects/590cd9c83a6749cc8b9f36a4972a59b0/users/e8e9fb82330749b582ac4522c4b8695f/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.731269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f40d0-e806-4c84-8181-dc9f77346427 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.736550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.737205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.737491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.737761 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] PATH_INFO: `/v3/projects/da0b85f5e518425680001d378b668aa9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.739108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da0b85f5e518425680001d378b668aa9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.740921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.742342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2b5ad-2eef-4fde-b296-2dc7c2206120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=590cd9c83a6749cc8b9f36a4972a59b0, user_id=e8e9fb82330749b582ac4522c4b8695f, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.745677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2b5ad-2eef-4fde-b296-2dc7c2206120 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.761081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1080/2161] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:26 2022] PUT /identity/v3/projects/590cd9c83a6749cc8b9f36a4972a59b0/users/e8e9fb82330749b582ac4522c4b8695f/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:26.771390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b08182-9204-4c9b-836c-07f73902377f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.771671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b08182-9204-4c9b-836c-07f73902377f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.773168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b08182-9204-4c9b-836c-07f73902377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.797041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da0b85f5e518425680001d378b668aa9', 'request_id': 'req-f466e5b7-a3de-4398-bc5d-14296f001eb2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.802767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da0b85f5e518425680001d378b668aa9', 'request_id': 'req-f466e5b7-a3de-4398-bc5d-14296f001eb2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.803290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-34b08182-9204-4c9b-836c-07f73902377f None None] MFA Rules not processed for user `e8e9fb82330749b582ac4522c4b8695f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:26.804669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-34b08182-9204-4c9b-836c-07f73902377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.809983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da0b85f5e518425680001d378b668aa9', 'request_id': 'req-f466e5b7-a3de-4398-bc5d-14296f001eb2'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.824897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:26.826320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] The token cache is being invalidate because project da0b85f5e518425680001d378b668aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:26.826583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.827228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.828815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1082/2162] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:26 2022] DELETE /identity/v3/projects/da0b85f5e518425680001d378b668aa9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:26.832840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.834304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.874578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1081/2163] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:26.886456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b08182-9204-4c9b-836c-07f73902377f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:26.888204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-34b08182-9204-4c9b-836c-07f73902377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:26.901806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466e5b7-a3de-4398-bc5d-14296f001eb2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.907108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.907779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.908081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.908341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] PATH_INFO: `/v3/users/dddaf956404c4df0892aeb03c1c221a6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.909537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dddaf956404c4df0892aeb03c1c221a6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.912060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.962486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b08182-9204-4c9b-836c-07f73902377f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:26.966794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:26.967465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:26.968046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:26.968046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:26.974309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dddaf956404c4df0892aeb03c1c221a6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.974979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dddaf956404c4df0892aeb03c1c221a6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.979690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:26.982198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:26.983121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dddaf956404c4df0892aeb03c1c221a6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.984156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dddaf956404c4df0892aeb03c1c221a6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.993522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f7f9fb84-c423-485f-be6f-c2ef58b3142d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dddaf956404c4df0892aeb03c1c221a6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:26.995336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1082/2164] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:27.022014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:27.036631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1083/2165] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:26 2022] DELETE /identity/v3/users/dddaf956404c4df0892aeb03c1c221a6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:27.360267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:27.377475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b6881-8e36-4cbc-a48e-fc899dd6a9f3 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:27.443341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:27.444056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.444322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.444600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] PATH_INFO: `/v3/projects/eef6c9a64571442ca4e64215ea1a8f1e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.445842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef6c9a64571442ca4e64215ea1a8f1e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:27.448026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:27.511219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef6c9a64571442ca4e64215ea1a8f1e', 'request_id': 'req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:27.521049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef6c9a64571442ca4e64215ea1a8f1e', 'request_id': 'req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:27.528252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef6c9a64571442ca4e64215ea1a8f1e', 'request_id': 'req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:27.533661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:27.551480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] The token cache is being invalidate because project eef6c9a64571442ca4e64215ea1a8f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:27.551804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:27.552559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ac9a0374-cac2-4169-9b09-f2aa6b7edbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:27.554157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1083/2166] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:27 2022] DELETE /identity/v3/projects/eef6c9a64571442ca4e64215ea1a8f1e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:27.560848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.561345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.561523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.586079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:27.587747 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:27.665688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1084/2167] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:27.668543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.668543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.668543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.690034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:27.691774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:27.701906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1084/2168] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:27.706572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:27.716661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c11d6-16f1-4c0d-80b1-419170094d8c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:27.730609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:27.731391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.731532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.731821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.732542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:27.732887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:27.734859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:27.748493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1085/2169] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:27.753528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:27.763065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f22d0-2042-47e2-b74a-1b3e7d002d40 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:27.779994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:27.780824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.781111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.782733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.783435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:27.785570 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:27.828853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1085/2170] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:27 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:27.833065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:27.848179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812270b-1f63-43ce-b443-da2d7f70b290 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:27.925393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:27.926167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.926542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.926943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.928357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:27.930074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:27.951898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1086/2171] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:27 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:27.956242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:27.966338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbe169-9707-4ce3-88df-bbfc26872249 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:27.970865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:27.971475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:27.971750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:27.972081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:27.972774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:27.974399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:27.984925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1086/2172] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:27.988965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:27.999150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f887ecf-b565-40a1-a6d3-13b1784df6d0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.004020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f17f6c-7a62-40ef-9bef-be87557c8e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.004635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f17f6c-7a62-40ef-9bef-be87557c8e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.004917 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f17f6c-7a62-40ef-9bef-be87557c8e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.005193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f17f6c-7a62-40ef-9bef-be87557c8e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.005886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f17f6c-7a62-40ef-9bef-be87557c8e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.007598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f17f6c-7a62-40ef-9bef-be87557c8e6b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.022504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1087/2173] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:28.023591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.033879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb607e0-9d44-402a-b01e-cebb038e1eff admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.038425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.039206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.039486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.040153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] PATH_INFO: `/v3/projects/adfb90b85d8c4ece95f08c5e6e464d87/users/e89b523fde32492bbb72f7be71c01e94/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.055222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfb90b85d8c4ece95f08c5e6e464d87, user_id=e89b523fde32492bbb72f7be71c01e94, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.057626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.073788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1087/2174] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:28 2022] PUT /identity/v3/projects/adfb90b85d8c4ece95f08c5e6e464d87/users/e89b523fde32492bbb72f7be71c01e94/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:28.085337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.085676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.085955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.116411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] MFA Rules not processed for user `e89b523fde32492bbb72f7be71c01e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:28.117597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:28.165496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:28.193581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1088/2175] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:28.198493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.263319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab785e-349f-457f-8302-154579aa0a2b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.268391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.269022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.269299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.269566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.270309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.272389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.300771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1088/2176] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:28 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:28.305569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.317948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b0c0d-f8a4-4c50-842a-140d119197be None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.378156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.378883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.379262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.379543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.380305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.382556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.411260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1089/2177] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:28 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:28.412625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.425796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49f281-3b0b-4bb4-98c8-86f6d9242b73 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.431865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.432494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.432831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.433150 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.433952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.435906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.451808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1089/2178] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:28.453251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.464299 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34272e8-ea2a-4bdc-906c-40ef25b7b72c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.468989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.469674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.469954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.470259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] PATH_INFO: `/v3/projects/5b0bec2c757c46fb9588cb35a8104267/users/6fba0747c8904e078ad38d9d7b9deed6/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.487480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0bec2c757c46fb9588cb35a8104267, user_id=6fba0747c8904e078ad38d9d7b9deed6, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.490197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.507195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1090/2179] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:28 2022] PUT /identity/v3/projects/5b0bec2c757c46fb9588cb35a8104267/users/6fba0747c8904e078ad38d9d7b9deed6/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:28.513091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.534680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e751d7a-c974-423c-9840-a6a576cc01fe admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.641234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391938d-9fbd-4a20-a222-4e452b1afb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.642048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391938d-9fbd-4a20-a222-4e452b1afb7b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.642397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391938d-9fbd-4a20-a222-4e452b1afb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.642736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391938d-9fbd-4a20-a222-4e452b1afb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.643916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391938d-9fbd-4a20-a222-4e452b1afb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.645863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391938d-9fbd-4a20-a222-4e452b1afb7b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.662807 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:28.675249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1090/2180] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:28.680873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:28.694985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170f86-e3c5-4393-92b1-6b02f382eadb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:28.706884 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:28.707679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.708070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.708818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] PATH_INFO: `/v3/projects/5b0bec2c757c46fb9588cb35a8104267/users/6fba0747c8904e078ad38d9d7b9deed6/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.711128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0bec2c757c46fb9588cb35a8104267, user_id=6fba0747c8904e078ad38d9d7b9deed6, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:28.714896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:28.726203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1091/2181] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:28 2022] PUT /identity/v3/projects/5b0bec2c757c46fb9588cb35a8104267/users/6fba0747c8904e078ad38d9d7b9deed6/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:28.802923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:28.803453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:28.807227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:28.839736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] MFA Rules not processed for user `6fba0747c8904e078ad38d9d7b9deed6`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:28.841358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:28.957746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1091/2182] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:29.003317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:29.005391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:29.102896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7edca-0ae6-4292-b3b3-bb8ad2504f27 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:29.108179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:29.108974 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:29.109410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:29.109695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:29.123008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:29.124433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:29.160060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1092/2183] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:29.273402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:29.275029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:29.287420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bab9e-eb93-4ba2-9a83-0117c1500d35 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:29.291467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd7f47-6bac-4562-a849-1a699efcf671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:29.292186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd7f47-6bac-4562-a849-1a699efcf671 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:29.292464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd7f47-6bac-4562-a849-1a699efcf671 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:29.292734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd7f47-6bac-4562-a849-1a699efcf671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:29.355495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd7f47-6bac-4562-a849-1a699efcf671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:29.356977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd7f47-6bac-4562-a849-1a699efcf671 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:29.365708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1092/2184] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:29.676222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:29.726507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:29.755948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01ab8-7e18-4f10-a202-766a5dfee5d5 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:29.761428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:29.762264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:29.762548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:29.762951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] PATH_INFO: `/v3/users/e89b523fde32492bbb72f7be71c01e94` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:29.764238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89b523fde32492bbb72f7be71c01e94)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:29.766512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:29.824682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89b523fde32492bbb72f7be71c01e94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:29.835271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89b523fde32492bbb72f7be71c01e94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:29.846329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89b523fde32492bbb72f7be71c01e94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:29.847362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89b523fde32492bbb72f7be71c01e94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:29.858234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89b523fde32492bbb72f7be71c01e94'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:29.886692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1093/2185] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:29 2022] DELETE /identity/v3/users/e89b523fde32492bbb72f7be71c01e94 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:29.888494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:29.889847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:29.890815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:29.917649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:29.918883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:30.021374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1093/2186] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:30.033617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:30.035131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:30.120164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194837c-17b3-424d-8584-8843303e2a0b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:30.132272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:30.133281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:30.133565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:30.133837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:30.166336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:30.168394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:30.179643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1094/2187] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:30.532731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:30.534427 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:30.554036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb922d0b-f0ab-489a-b62d-ae9274397ddb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:30.568119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:30.569022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:30.569905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:30.570293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] PATH_INFO: `/v3/projects/adfb90b85d8c4ece95f08c5e6e464d87` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:30.571658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] RBAC: Authorizing `identity:delete_project(project_id=adfb90b85d8c4ece95f08c5e6e464d87)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:30.575050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:30.619625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:30.668109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adfb90b85d8c4ece95f08c5e6e464d87', 'request_id': 'req-6695d07e-3ca4-4b86-9747-b373ca7c254e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.675278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adfb90b85d8c4ece95f08c5e6e464d87', 'request_id': 'req-6695d07e-3ca4-4b86-9747-b373ca7c254e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.683438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adfb90b85d8c4ece95f08c5e6e464d87', 'request_id': 'req-6695d07e-3ca4-4b86-9747-b373ca7c254e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.712247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] The token cache is being invalidate because project adfb90b85d8c4ece95f08c5e6e464d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:30.713740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.716017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.721037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1094/2188] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:30 2022] DELETE /identity/v3/projects/adfb90b85d8c4ece95f08c5e6e464d87 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:30.725340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:30.739081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63204c-afc4-4f8d-a7cc-db1e34fc554b service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:30.815398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:30.816099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:30.816551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:30.816835 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] PATH_INFO: `/v3/users/6fba0747c8904e078ad38d9d7b9deed6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:30.818140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fba0747c8904e078ad38d9d7b9deed6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:30.820593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:30.823474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:30.878695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fba0747c8904e078ad38d9d7b9deed6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.879335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fba0747c8904e078ad38d9d7b9deed6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.886714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fba0747c8904e078ad38d9d7b9deed6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.887439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fba0747c8904e078ad38d9d7b9deed6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.895681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-31e0b608-88d1-475a-8eed-e42b2ec642e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fba0747c8904e078ad38d9d7b9deed6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:30.915351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1095/2189] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:30 2022] DELETE /identity/v3/users/6fba0747c8904e078ad38d9d7b9deed6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:31.477291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:31.479307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:31.492836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695d07e-3ca4-4b86-9747-b373ca7c254e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:31.585169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:31.585967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:31.586239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:31.586994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] PATH_INFO: `/v3/projects/5b0bec2c757c46fb9588cb35a8104267` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:31.587793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0bec2c757c46fb9588cb35a8104267)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:31.589728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:31.653789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0bec2c757c46fb9588cb35a8104267', 'request_id': 'req-02211e8a-7ad8-44c6-9942-6c988f70b15b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:31.661520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0bec2c757c46fb9588cb35a8104267', 'request_id': 'req-02211e8a-7ad8-44c6-9942-6c988f70b15b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:31.669368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0bec2c757c46fb9588cb35a8104267', 'request_id': 'req-02211e8a-7ad8-44c6-9942-6c988f70b15b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:31.685351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] The token cache is being invalidate because project 5b0bec2c757c46fb9588cb35a8104267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:31.685658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:31.686499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-02211e8a-7ad8-44c6-9942-6c988f70b15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:31.688226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1095/2190] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:31 2022] DELETE /identity/v3/projects/5b0bec2c757c46fb9588cb35a8104267 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:31.693556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:31.693843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:31.694120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:31.714191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:31.715399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:31.793267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1096/2191] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:31.801192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:31.803798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:31.803798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:31.822505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:31.823753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:31.839013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1096/2192] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:31.840385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:31.851724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42561ce0-bfa0-4096-9c69-7856b8a3e81a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:31.858222 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:31.858984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:31.859364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:31.859729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:31.860557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:31.860946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:31.862647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:31.879242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1097/2193] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:14:31.880760 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:31.890746 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c445c-52b0-4cbf-87cb-1dced0cfbc57 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:31.895922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:31.896545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:31.896826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:31.897097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:31.897786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:31.899740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:31.929412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1097/2194] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:31 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:31.934625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:31.947172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f368-c2c2-488a-9b8b-a206f1713675 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.024822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6a7b0-7952-4ce6-b14c-8fe3861f04e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.025823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6a7b0-7952-4ce6-b14c-8fe3861f04e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.025930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6a7b0-7952-4ce6-b14c-8fe3861f04e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.026317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6a7b0-7952-4ce6-b14c-8fe3861f04e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.027013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6a7b0-7952-4ce6-b14c-8fe3861f04e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.029157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6a7b0-7952-4ce6-b14c-8fe3861f04e2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.057117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:32.066915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1098/2195] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:31 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:32.068580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.072708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b84ce-e02a-4c06-a17e-de269077ca5a None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.073282 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b84ce-e02a-4c06-a17e-de269077ca5a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.073728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b84ce-e02a-4c06-a17e-de269077ca5a None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.076012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1099/2196] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 05:14:32.094766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f5061-feb7-4359-a6b1-dafe1fc031f0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.100344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.100979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.101295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.101572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.102274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.103938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.106328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.106465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:32.107323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.107758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.123453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1098/2197] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:32.127815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.137034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:32.138485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.146146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31dbcb-3c91-4669-bcce-0e9ff8b64b34 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.160722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.161537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.161831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.162254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.162971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.165163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.186409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1099/2198] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:32.187449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.189665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.205915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab451f-85b1-4e8f-813b-72664e8c55a8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.211013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.211753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.212054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.212241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] PATH_INFO: `/v3/users/28432f2e1dea486cb163fa2dab6e443f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.213429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28432f2e1dea486cb163fa2dab6e443f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.215342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.228513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1100/2199] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:14:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:32.229439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.251531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0194f-450a-4395-979a-8e203a94f194 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.257017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.257804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.258114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.258424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] PATH_INFO: `/v3/projects/83b28a5887a84cd395282029952f0b25/users/2d198fe34157430cba464fe85954872b/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.273493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28432f2e1dea486cb163fa2dab6e443f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.274184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28432f2e1dea486cb163fa2dab6e443f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.275532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b28a5887a84cd395282029952f0b25, user_id=2d198fe34157430cba464fe85954872b, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.278687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.281146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28432f2e1dea486cb163fa2dab6e443f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.281796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28432f2e1dea486cb163fa2dab6e443f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.293379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a63ba386-cc26-4a58-9496-7a958d5bd9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28432f2e1dea486cb163fa2dab6e443f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.294430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1101/2200] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:32 2022] PUT /identity/v3/projects/83b28a5887a84cd395282029952f0b25/users/2d198fe34157430cba464fe85954872b/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:32.295404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.297993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.313837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1100/2201] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:32 2022] DELETE /identity/v3/users/28432f2e1dea486cb163fa2dab6e443f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:32.315106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.315391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.315670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.344930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] MFA Rules not processed for user `2d198fe34157430cba464fe85954872b`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:32.346173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.384446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73057-8681-4b10-897b-171590579778 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.435383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1101/2202] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:32.455524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.457308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.465385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.466102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.466386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.466754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.468604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.480467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.482511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.494810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1102/2203] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:32.514932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.516439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.526085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:32.526475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:32.531387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4baef-8b84-4d8c-a358-9aced0413ee8 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.536112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b67ef-ccc5-483f-93c5-e65aeebeb1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.537398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b67ef-ccc5-483f-93c5-e65aeebeb1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.537603 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b67ef-ccc5-483f-93c5-e65aeebeb1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.537886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b67ef-ccc5-483f-93c5-e65aeebeb1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.550769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b67ef-ccc5-483f-93c5-e65aeebeb1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.552830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b67ef-ccc5-483f-93c5-e65aeebeb1b4 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.566271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1102/2204] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:32.613949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297e26b-5946-4322-9849-c32e564ffe1d service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.618763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.619485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.619859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.620313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] PATH_INFO: `/v3/projects/08e98cefc4d147bba944a1cf82f2e772` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.621541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e98cefc4d147bba944a1cf82f2e772)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.623722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:32.698069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e98cefc4d147bba944a1cf82f2e772', 'request_id': 'req-0b60d98e-07d9-4f39-87ff-79a716748ffc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.704449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e98cefc4d147bba944a1cf82f2e772', 'request_id': 'req-0b60d98e-07d9-4f39-87ff-79a716748ffc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.712482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e98cefc4d147bba944a1cf82f2e772', 'request_id': 'req-0b60d98e-07d9-4f39-87ff-79a716748ffc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.735666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] The token cache is being invalidate because project 08e98cefc4d147bba944a1cf82f2e772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:32.735991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.737582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-0b60d98e-07d9-4f39-87ff-79a716748ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:32.740238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1103/2205] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:32 2022] DELETE /identity/v3/projects/08e98cefc4d147bba944a1cf82f2e772 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:32.745415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.745783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.746187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.785462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:32.787073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.909593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1103/2206] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:32.919386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.920429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.920604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.949033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:32.950546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:32.967756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1104/2207] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:32.972034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:32.981986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c89fd-37ea-4132-a83d-2883e84630ee None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:32.990991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:32.991656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:32.992012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:32.992290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:32.993326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:32.993635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:32.995795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.011820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1104/2208] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:33.019404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.029391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bed8c0-6a2a-43c1-a4cb-92ec218e26b3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.034124 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.034772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.035100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.035362 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.036175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.038185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.067813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1105/2209] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:33 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:33.070876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.081789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac5b17-3244-4efc-be78-926f4b13abb1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.141634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.142311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.142552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.142830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.143495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.145460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.168876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1105/2210] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:33 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:33.173183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.183496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da700cdf-38a1-42fa-8eb2-0b1ec1c88c3c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.188090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.188697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.188978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.189251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.189956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.191559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.201602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1106/2211] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:33.205674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.216007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f438a7-9d7f-4dbf-aa77-2fccf06a6440 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.221295 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcaff1-5d1a-4aaf-a6f8-a54a89d16487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.221925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcaff1-5d1a-4aaf-a6f8-a54a89d16487 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.222361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcaff1-5d1a-4aaf-a6f8-a54a89d16487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.222480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcaff1-5d1a-4aaf-a6f8-a54a89d16487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.223344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcaff1-5d1a-4aaf-a6f8-a54a89d16487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.225507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcaff1-5d1a-4aaf-a6f8-a54a89d16487 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.238284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1106/2212] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:33.242814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.253732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39e6c1-c479-4898-8139-e6307a0fac0e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.259012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.259693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.260496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.260788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] PATH_INFO: `/v3/projects/43ae83fbfbf84574b3c3b3b15230aaec/users/9770fd7cc69b4714a163ba69d7cfbb8c/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.276347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ae83fbfbf84574b3c3b3b15230aaec, user_id=9770fd7cc69b4714a163ba69d7cfbb8c, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.279030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.292463 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1107/2213] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:33 2022] PUT /identity/v3/projects/43ae83fbfbf84574b3c3b3b15230aaec/users/9770fd7cc69b4714a163ba69d7cfbb8c/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:33.303119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.303403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.303671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.335755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] MFA Rules not processed for user `9770fd7cc69b4714a163ba69d7cfbb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:33.337023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:33.386731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:33.408164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1107/2214] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:33.413662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.447201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:33.479533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5f7fd-0967-4360-a0b2-f13204490098 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.485461 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.486091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.486379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.486659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.487575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.490891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.525060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1108/2215] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:33 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:33.530315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.542073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6feac5-0fcf-4007-9e38-4ad4620e011f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.598709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.599399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.599681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.600003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.600649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.602732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.624262 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1108/2216] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:33 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:33.628666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.639097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec7478-a07e-40e4-a5b1-b4dee82ccd07 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.643979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.644596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.644879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.645155 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.645862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.647597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.658336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1109/2217] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:33.662532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.673313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c6d09-e02c-4499-b986-ad59c07b4b98 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.679078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.679860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.680143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.680413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] PATH_INFO: `/v3/projects/a82ac03a03c0417c8831321e38e35b40/users/8fc61ec62f9a4a5fa6e7e485c8159523/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.696267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82ac03a03c0417c8831321e38e35b40, user_id=8fc61ec62f9a4a5fa6e7e485c8159523, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.698941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.714580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1109/2218] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:33 2022] PUT /identity/v3/projects/a82ac03a03c0417c8831321e38e35b40/users/8fc61ec62f9a4a5fa6e7e485c8159523/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:33.718617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.728567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaa67a-9fcc-46d3-a57d-ddc1aa4a2bae admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.785938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbdd15-eaec-4ed0-b570-e8e22797858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.786476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdd15-eaec-4ed0-b570-e8e22797858c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.786754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdd15-eaec-4ed0-b570-e8e22797858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.787025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdd15-eaec-4ed0-b570-e8e22797858c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.787712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbdd15-eaec-4ed0-b570-e8e22797858c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.789392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbdd15-eaec-4ed0-b570-e8e22797858c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.799323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1110/2219] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:33.804584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:33.820292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59efd0-cda8-48dc-8ece-f7cc06bedc9d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:33.828266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:33.829296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.829639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.829991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] PATH_INFO: `/v3/projects/a82ac03a03c0417c8831321e38e35b40/users/8fc61ec62f9a4a5fa6e7e485c8159523/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.834061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82ac03a03c0417c8831321e38e35b40, user_id=8fc61ec62f9a4a5fa6e7e485c8159523, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:33.837536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:33.848767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1110/2220] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:33 2022] PUT /identity/v3/projects/a82ac03a03c0417c8831321e38e35b40/users/8fc61ec62f9a4a5fa6e7e485c8159523/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:33.860544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:33.860839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:33.861117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:33.894074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] MFA Rules not processed for user `8fc61ec62f9a4a5fa6e7e485c8159523`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:33.895297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:33.990554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1111/2221] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:34.041991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:34.043731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:34.114319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d20b9e-57b1-45c9-ad44-80da2cc5a09f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:34.118960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:34.119632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:34.120008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:34.120256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:34.130250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:34.132646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:34.146434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1111/2222] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:34.266574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:34.268022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:34.282675 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a3945-d722-47f8-a839-f1952e857e3e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:34.286347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db49e-048e-48ac-8d74-160b8a5f91f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:34.286977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db49e-048e-48ac-8d74-160b8a5f91f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:34.287257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db49e-048e-48ac-8d74-160b8a5f91f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:34.287531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db49e-048e-48ac-8d74-160b8a5f91f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:34.309075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:34.351206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799db49e-048e-48ac-8d74-160b8a5f91f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:34.352729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799db49e-048e-48ac-8d74-160b8a5f91f2 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:34.362310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1112/2223] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:40.284391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:40.336291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:40.356413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd632ee-88f4-4d52-9cea-a1ddb5893b85 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:40.360990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:40.361661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:40.361920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:40.362193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] PATH_INFO: `/v3/users/9770fd7cc69b4714a163ba69d7cfbb8c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:40.363376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9770fd7cc69b4714a163ba69d7cfbb8c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:40.365317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:40.413884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9770fd7cc69b4714a163ba69d7cfbb8c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:40.414523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9770fd7cc69b4714a163ba69d7cfbb8c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:40.421716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9770fd7cc69b4714a163ba69d7cfbb8c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:40.422328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9770fd7cc69b4714a163ba69d7cfbb8c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:40.430773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9770fd7cc69b4714a163ba69d7cfbb8c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:40.448563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1112/2224] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:40 2022] DELETE /identity/v3/users/9770fd7cc69b4714a163ba69d7cfbb8c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:40.453184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:40.453655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:40.453872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:40.472737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:40.473920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:40.536377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1113/2225] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:40.546699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:40.548374 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:40.608678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bff67-f233-431d-9c1d-55783455d842 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:40.612518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:40.613229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:40.613413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:40.613688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:40.623335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:40.624812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:40.634744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1113/2226] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:40.928824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:40.930300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:40.942526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b75a21-3762-45d3-86f8-b5b6d2b1e9b2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:40.948565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:40.949225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:40.949501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:40.949778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] PATH_INFO: `/v3/projects/43ae83fbfbf84574b3c3b3b15230aaec` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:40.958953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ae83fbfbf84574b3c3b3b15230aaec)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:40.960949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:41.022862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ae83fbfbf84574b3c3b3b15230aaec', 'request_id': 'req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.028503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ae83fbfbf84574b3c3b3b15230aaec', 'request_id': 'req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.034720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ae83fbfbf84574b3c3b3b15230aaec', 'request_id': 'req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.048881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] The token cache is being invalidate because project 43ae83fbfbf84574b3c3b3b15230aaec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:41.049202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.049955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.051558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1114/2227] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:40 2022] DELETE /identity/v3/projects/43ae83fbfbf84574b3c3b3b15230aaec => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:41.055523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:41.065588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28af8d4-6182-4070-957a-77cbd4ab8ab6 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:41.126166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:41.126839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.127126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.127391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] PATH_INFO: `/v3/users/8fc61ec62f9a4a5fa6e7e485c8159523` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.128624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc61ec62f9a4a5fa6e7e485c8159523)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:41.130539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:41.135346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:41.190878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc61ec62f9a4a5fa6e7e485c8159523'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.191628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc61ec62f9a4a5fa6e7e485c8159523'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.199218 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc61ec62f9a4a5fa6e7e485c8159523'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.199905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc61ec62f9a4a5fa6e7e485c8159523'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.209414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-1541f57d-3951-41bf-be7d-2bf5bd9e3f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc61ec62f9a4a5fa6e7e485c8159523'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.226876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1114/2228] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:41 2022] DELETE /identity/v3/users/8fc61ec62f9a4a5fa6e7e485c8159523 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:41.510957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:41.512599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:41.543495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc960386-91fe-4b3b-941d-fdcb9f3bcc25 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:41.589305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:41.603442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:41.604137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.604422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.604700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] PATH_INFO: `/v3/projects/a82ac03a03c0417c8831321e38e35b40` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.605864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82ac03a03c0417c8831321e38e35b40)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:41.607804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:41.661803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82ac03a03c0417c8831321e38e35b40', 'request_id': 'req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.667825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82ac03a03c0417c8831321e38e35b40', 'request_id': 'req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.674298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82ac03a03c0417c8831321e38e35b40', 'request_id': 'req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.689045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] The token cache is being invalidate because project a82ac03a03c0417c8831321e38e35b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:41.689370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.690078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5958bdb0-01df-4dd4-bd5c-4e2713c980f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:41.692128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1115/2229] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:41 2022] DELETE /identity/v3/projects/a82ac03a03c0417c8831321e38e35b40 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:41.697307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.697808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.697808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.717496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:41.718728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:41.784358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1115/2230] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:41.792642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.793436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.793721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.814782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:41.816025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:41.825082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1116/2231] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:41.829576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:41.839456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e3376-2933-490c-b8c8-33cf125a5ab1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:41.844757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:41.845377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.845658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.845932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.846640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:41.846949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:41.848586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:41.860359 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1116/2232] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:14:41.864821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:41.874782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d500f-850d-4468-966b-0a38ec8c3ba9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:41.880818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:41.882119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.882404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.883070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.883355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:41.885383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:41.919876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1117/2233] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:41.922533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:41.932859 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26f2a8-d796-4345-af55-f33ac5c44a98 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:41.991579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:41.992236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:41.992515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:41.992793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:41.993530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:41.995475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.021426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:42.021629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1117/2234] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:41 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:42.025632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.036685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e33e-d3c5-4858-ae9c-621d54f53f83 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.041780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.042425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.042709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.042987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.043683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.045420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.060129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1118/2235] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:42.061913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.072618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dbf5b-d92f-4719-9afe-16394b6206cb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.077605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f40a8-0a91-4b4c-98b9-1054af4d1604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.078267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f40a8-0a91-4b4c-98b9-1054af4d1604 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.078558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f40a8-0a91-4b4c-98b9-1054af4d1604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.078828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f40a8-0a91-4b4c-98b9-1054af4d1604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.079536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f40a8-0a91-4b4c-98b9-1054af4d1604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.081313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f40a8-0a91-4b4c-98b9-1054af4d1604 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.092926 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1118/2236] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:42.097244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.107778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7041bab-dc97-4399-8ec0-e843916e85e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.112403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.113380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.113660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.113936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] PATH_INFO: `/v3/projects/2a5e23d151c943b78dec682dae9cdc4f/users/a5a0ce0c51104920b8804aa19fbd626c/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.128776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5e23d151c943b78dec682dae9cdc4f, user_id=a5a0ce0c51104920b8804aa19fbd626c, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.131485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.146432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1119/2237] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:42 2022] PUT /identity/v3/projects/2a5e23d151c943b78dec682dae9cdc4f/users/a5a0ce0c51104920b8804aa19fbd626c/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:42.157173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.157566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.157904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.189188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] MFA Rules not processed for user `a5a0ce0c51104920b8804aa19fbd626c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:42.190459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:42.267166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1119/2238] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:42.272881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.347543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff43708-0464-43da-aa0c-1dc3feee0d07 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.352552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.353514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.353865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.354208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.354987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.357035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.381765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:42.389271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1120/2239] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:42 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:42.393798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.405453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ecb58-a03e-43d8-bc37-e065ae213732 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.468059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.468727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.469009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.469286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.469953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.471900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.493553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1120/2240] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:42 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:42.497772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.508397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a99b61-4099-4e56-898d-6872821b0a14 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.513165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.513800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.514054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.514354 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.515052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.516689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.530141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1121/2241] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:42.531982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.542756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e1e30-10b8-4c55-bfbc-5da29787dfab admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.548143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.549031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.549311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.549579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] PATH_INFO: `/v3/projects/8cbe14b8b1634119b4e1547a179814ac/users/82c1f2a26ce84806b59ce1171176ee49/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.565425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbe14b8b1634119b4e1547a179814ac, user_id=82c1f2a26ce84806b59ce1171176ee49, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.568190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.584175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1121/2242] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:42 2022] PUT /identity/v3/projects/8cbe14b8b1634119b4e1547a179814ac/users/82c1f2a26ce84806b59ce1171176ee49/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:42.588402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.599105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb02478-4f45-441f-8e07-894b7556f3e0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.671625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ff30d-71d3-4045-b89e-2442fba674d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.672591 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff30d-71d3-4045-b89e-2442fba674d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.672928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff30d-71d3-4045-b89e-2442fba674d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.673397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff30d-71d3-4045-b89e-2442fba674d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.674161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ff30d-71d3-4045-b89e-2442fba674d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.676165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ff30d-71d3-4045-b89e-2442fba674d8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.691078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1122/2243] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:42.692373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.703008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e335b3-bf56-40be-92e4-8325de50f386 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.707672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.708451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.708802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.709152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] PATH_INFO: `/v3/projects/8cbe14b8b1634119b4e1547a179814ac/users/82c1f2a26ce84806b59ce1171176ee49/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.711246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbe14b8b1634119b4e1547a179814ac, user_id=82c1f2a26ce84806b59ce1171176ee49, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.714084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.725254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1122/2244] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:42 2022] PUT /identity/v3/projects/8cbe14b8b1634119b4e1547a179814ac/users/82c1f2a26ce84806b59ce1171176ee49/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:42.736656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.736957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.737261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.752120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.754601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:42.768176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] MFA Rules not processed for user `82c1f2a26ce84806b59ce1171176ee49`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:42.769597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:42.798362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:42.832707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a1fd2-d28a-4acd-8417-b067c927905a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.837364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.838009 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.838297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.838574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] PATH_INFO: `/v3/users/14ea77c59f54488ea9fe28bb821b5dea` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.839759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ea77c59f54488ea9fe28bb821b5dea)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.841683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1123/2245] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:42.841806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:42.852592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:42.853953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:42.890228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ea77c59f54488ea9fe28bb821b5dea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:42.890890 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ea77c59f54488ea9fe28bb821b5dea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:42.898426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ea77c59f54488ea9fe28bb821b5dea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:42.899053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ea77c59f54488ea9fe28bb821b5dea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:42.906711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ea77c59f54488ea9fe28bb821b5dea'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:42.913663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3558fa-b272-41c6-aba6-67de0ba780e9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:42.918388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:42.919000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:42.919283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:42.919708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:42.921973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:42.925879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1123/2246] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:42 2022] DELETE /identity/v3/users/14ea77c59f54488ea9fe28bb821b5dea => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:42.952232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:42.994251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:42.995741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:43.005576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1124/2247] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:14:43.112900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:43.114517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:43.195514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49535a-4d6d-42c4-996e-89804abf84f2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:43.200180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:43.200821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:43.201105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:43.201377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] PATH_INFO: `/v3/projects/71fa01660bbf481c810ecb6374e4bdbf` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:43.202527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fa01660bbf481c810ecb6374e4bdbf)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:43.204475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:43.265461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fa01660bbf481c810ecb6374e4bdbf', 'request_id': 'req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.267814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:43.273626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fa01660bbf481c810ecb6374e4bdbf', 'request_id': 'req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.284271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fa01660bbf481c810ecb6374e4bdbf', 'request_id': 'req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.300698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] The token cache is being invalidate because project 71fa01660bbf481c810ecb6374e4bdbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:43.301070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.302520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.307236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1124/2248] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:43 2022] DELETE /identity/v3/projects/71fa01660bbf481c810ecb6374e4bdbf => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:43.309092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:43.320279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7d462-ef67-4fed-9014-ffe13a5659e3 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:43.375586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:43.376253 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:43.376532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:43.376806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] PATH_INFO: `/v3/users/cfc240246f624dbca3f006f423865ca9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:43.377953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc240246f624dbca3f006f423865ca9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:43.379899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:43.428206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc240246f624dbca3f006f423865ca9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.428860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc240246f624dbca3f006f423865ca9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.435878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc240246f624dbca3f006f423865ca9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.436512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc240246f624dbca3f006f423865ca9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.445013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ba55da15-c3bc-45c1-abca-92832d469c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc240246f624dbca3f006f423865ca9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.464666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1125/2249] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:43 2022] DELETE /identity/v3/users/cfc240246f624dbca3f006f423865ca9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:43.680384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:43.681964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:43.696752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f024ca1-04a0-43fa-9a1a-fc85ec31f5e6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:43.759053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:43.759984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:43.760290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:43.760565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] PATH_INFO: `/v3/projects/a9961b4b8ece496992424ee268a4d57a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:43.761734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9961b4b8ece496992424ee268a4d57a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:43.763628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:43.820499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9961b4b8ece496992424ee268a4d57a', 'request_id': 'req-b229b3b3-ed01-43b3-b007-97b965273565'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.826694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9961b4b8ece496992424ee268a4d57a', 'request_id': 'req-b229b3b3-ed01-43b3-b007-97b965273565'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.832927 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9961b4b8ece496992424ee268a4d57a', 'request_id': 'req-b229b3b3-ed01-43b3-b007-97b965273565'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.847828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] The token cache is being invalidate because project a9961b4b8ece496992424ee268a4d57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:43.848116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.848786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b229b3b3-ed01-43b3-b007-97b965273565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:43.850372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1125/2250] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:43 2022] DELETE /identity/v3/projects/a9961b4b8ece496992424ee268a4d57a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:43.855695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:43.856196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:43.856306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:43.878331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:43.879472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:43.920654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:43.942216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1126/2251] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:43.950263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:43.950552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:43.950830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:43.972928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:43.974753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:43.987550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1126/2252] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:43.992430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.008211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ddb13-d04f-45e1-80ae-d9ec73f41e86 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.014181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.014924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.015207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.015477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.016266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:44.016552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.018306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.032655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1127/2253] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:44.037053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.046997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aadd83-dd13-4cdc-b2b7-3861e9b0af79 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.051586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.052265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.052515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.052762 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.053556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.055433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.084449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1127/2254] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:44 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:44.088692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.099184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d256941-535e-4bfa-a9c7-632ea0b6ab09 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.158511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.159126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.159407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.159681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.160376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.162333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.184868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1128/2255] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:44 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:44.189129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.199449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e5b87-8ec9-4955-afee-a798c14f2298 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.204310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.204942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.205201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.205471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.206171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.207803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.218249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1128/2256] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:44.223888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.234511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8af8-61ed-4184-9543-e73e9f4f5f0c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.240016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7c92-839d-44ef-9aae-3617e03a7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.240628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7c92-839d-44ef-9aae-3617e03a7334 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.240935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7c92-839d-44ef-9aae-3617e03a7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.241205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7c92-839d-44ef-9aae-3617e03a7334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.241901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab7c92-839d-44ef-9aae-3617e03a7334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.243502 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab7c92-839d-44ef-9aae-3617e03a7334 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.254870 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1129/2257] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:44.259346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.271818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cda749-5a41-41fc-b2de-44601cc16763 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.277276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.278022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.278305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.278581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] PATH_INFO: `/v3/projects/888222be148b4022a037b3855c7cb57a/users/52d61d96a51e484587e499f1e8b03aea/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.294390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888222be148b4022a037b3855c7cb57a, user_id=52d61d96a51e484587e499f1e8b03aea, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.297119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.312083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1129/2258] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:44 2022] PUT /identity/v3/projects/888222be148b4022a037b3855c7cb57a/users/52d61d96a51e484587e499f1e8b03aea/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:44.324640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.324932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.325206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.355721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] MFA Rules not processed for user `52d61d96a51e484587e499f1e8b03aea`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:44.356970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:44.422633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1130/2259] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:44.427477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.437548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:44.502008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ef5de-b130-4074-addc-2fa4b49d5c09 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.507329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.508011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.508290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.508568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.509351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.511325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.546583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1130/2260] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:44 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:14:44.548507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.560107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b49c81-ca17-406c-b6e8-4416d420dd0f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.598911 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:44.625168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.625850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.626168 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.626408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.627093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.629059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.655797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1131/2261] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:44 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:44.657682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.669481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b62ff3-c5bb-4521-91af-6cbcbd03f62d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.675876 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.676432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.676706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.676975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.677897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.679555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.693949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1131/2262] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:44.698215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.708933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a194f-b1f4-4a3f-9111-1b57c66db979 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.713850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.715860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.716166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.717182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] PATH_INFO: `/v3/projects/92cc65dc26844cd9be77104437f56989/users/4715c241492148aca66af4beb300bc3f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.735152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92cc65dc26844cd9be77104437f56989, user_id=4715c241492148aca66af4beb300bc3f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.737844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.753772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1132/2263] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:44 2022] PUT /identity/v3/projects/92cc65dc26844cd9be77104437f56989/users/4715c241492148aca66af4beb300bc3f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:44.758185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.768852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceeaf64-2a7b-4da3-994d-626c7d5f4d91 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.829460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd68302-5d67-4c27-af44-9709fad7cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.830095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd68302-5d67-4c27-af44-9709fad7cac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.830433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd68302-5d67-4c27-af44-9709fad7cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.830721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd68302-5d67-4c27-af44-9709fad7cac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.831438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd68302-5d67-4c27-af44-9709fad7cac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.833258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd68302-5d67-4c27-af44-9709fad7cac5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.847839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1132/2264] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:44.849624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:44.859899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f2ed-edcd-4c43-80b4-a0caf5e25b11 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:44.864652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:44.865588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.865821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.866162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] PATH_INFO: `/v3/projects/92cc65dc26844cd9be77104437f56989/users/4715c241492148aca66af4beb300bc3f/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.868361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92cc65dc26844cd9be77104437f56989, user_id=4715c241492148aca66af4beb300bc3f, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:44.871305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:44.886162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1133/2265] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:44 2022] PUT /identity/v3/projects/92cc65dc26844cd9be77104437f56989/users/4715c241492148aca66af4beb300bc3f/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:44.896657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:44.896974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:44.897299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:44.926913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] MFA Rules not processed for user `4715c241492148aca66af4beb300bc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:44.928508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:45.000561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1133/2266] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:45.014048 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:45.016131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:45.079515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94843f0-5f08-4cb9-bb60-84315a5cf7e5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:45.083565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:45.084212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:45.084494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:45.084792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:45.148948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:45.150646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:45.160627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1134/2267] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:51.913864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:51.915886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:51.923353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:51.930399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98525-0825-4cdb-abcf-4344ad5e0c74 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:51.934477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:51.935121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:51.935414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:51.935686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:51.937498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:52.003880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:52.005737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61108a-d467-4321-87f7-8fc7f0ec9946 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:52.022360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1134/2268] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:53.142317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:53.192994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:53.212129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e458e-7d1b-412c-a5d5-4434b0489c1d service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:53.216826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:53.217553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:53.217843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:53.218121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] PATH_INFO: `/v3/users/52d61d96a51e484587e499f1e8b03aea` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:53.219430 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d61d96a51e484587e499f1e8b03aea)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:53.221388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:53.276146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d61d96a51e484587e499f1e8b03aea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.276789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d61d96a51e484587e499f1e8b03aea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.283519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d61d96a51e484587e499f1e8b03aea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.284148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d61d96a51e484587e499f1e8b03aea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.292722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d61d96a51e484587e499f1e8b03aea'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.311171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1135/2269] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:53 2022] DELETE /identity/v3/users/52d61d96a51e484587e499f1e8b03aea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:53.315947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:53.316402 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:53.316630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:53.339276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:53.341030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:53.407618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1135/2270] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:53.417840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:53.419318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:53.479005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f1a07-389f-4d5f-883e-18f3d32a29f3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:53.483748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:53.484403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:53.484713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:53.484988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:53.494965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:53.496457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:53.507789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1136/2271] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:53.787074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:53.788645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:53.801521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddeb2ad-1f3f-482a-a6bf-cbb6b726e19b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:53.807170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:53.807854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:53.808137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:53.808416 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] PATH_INFO: `/v3/projects/888222be148b4022a037b3855c7cb57a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:53.809611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=888222be148b4022a037b3855c7cb57a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:53.812582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:53.862163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888222be148b4022a037b3855c7cb57a', 'request_id': 'req-e0537940-6579-473e-a4fa-8839f82c94c4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.867517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888222be148b4022a037b3855c7cb57a', 'request_id': 'req-e0537940-6579-473e-a4fa-8839f82c94c4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.873563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888222be148b4022a037b3855c7cb57a', 'request_id': 'req-e0537940-6579-473e-a4fa-8839f82c94c4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.887138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] The token cache is being invalidate because project 888222be148b4022a037b3855c7cb57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:53.887425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.888083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:53.890212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1136/2272] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:53 2022] DELETE /identity/v3/projects/888222be148b4022a037b3855c7cb57a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:53.894373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:53.904840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15c3f0-32ee-4856-9f9d-bcf671f57312 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:53.968687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:53.969333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:53.969614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:53.969888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] PATH_INFO: `/v3/users/4715c241492148aca66af4beb300bc3f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:53.971073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4715c241492148aca66af4beb300bc3f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:53.973049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:53.978473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:54.022671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4715c241492148aca66af4beb300bc3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.023327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4715c241492148aca66af4beb300bc3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.030214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4715c241492148aca66af4beb300bc3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.030837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4715c241492148aca66af4beb300bc3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.039210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-199398ef-0482-453a-9def-7fe024f95f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4715c241492148aca66af4beb300bc3f'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.059017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1137/2273] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:53 2022] DELETE /identity/v3/users/4715c241492148aca66af4beb300bc3f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:54.418509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.420217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:54.434008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0537940-6579-473e-a4fa-8839f82c94c4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.452131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:54.492595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.493239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.493520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.493789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] PATH_INFO: `/v3/projects/92cc65dc26844cd9be77104437f56989` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.494931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92cc65dc26844cd9be77104437f56989)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:54.496881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:54.544349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92cc65dc26844cd9be77104437f56989', 'request_id': 'req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.549956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92cc65dc26844cd9be77104437f56989', 'request_id': 'req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.555994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92cc65dc26844cd9be77104437f56989', 'request_id': 'req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.569800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] The token cache is being invalidate because project 92cc65dc26844cd9be77104437f56989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:54.570096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.570722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-db8f0b6d-154b-4abe-8c5b-3b0dcf19bbf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:54.572269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1137/2274] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:54 2022] DELETE /identity/v3/projects/92cc65dc26844cd9be77104437f56989 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:54.578739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.579197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.579284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.598720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:54.599961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:54.661803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1138/2275] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:54.674204 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.674496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.674772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.695010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:54.696236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:54.705065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1138/2276] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:54.709327 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.719037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3fd1d-9729-403b-b05b-251e97082ca7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.724735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.725402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.725682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.725955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.726662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:14:54.726967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:54.728700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:54.740703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1139/2277] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:14:54.745122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.754714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350b738-63ff-4be0-acd6-2aee0de087e1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.760842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.761487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.761765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.762036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.762703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:54.764672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:54.795085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1139/2278] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:14:54 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:54.799404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.809778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456b3fb-ac77-4ee1-8f78-764891190a69 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.869250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.869868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.870140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.870407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.871078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:54.873012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:54.896405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1140/2279] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:14:54 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:14:54.900814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.911902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b020650-6db5-497e-ba30-c0f9794687c5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.917763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.918382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.918663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.918935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.919641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:54.921405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:54.931623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1140/2280] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:14:54.935654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.940838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:54.946838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61df61-d8f0-4b3b-aa57-4088b1906d5c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.952979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5059b4b-7925-4004-a039-37d81c854fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.953590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059b4b-7925-4004-a039-37d81c854fde admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.953876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059b4b-7925-4004-a039-37d81c854fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.954154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5059b4b-7925-4004-a039-37d81c854fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:54.954860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5059b4b-7925-4004-a039-37d81c854fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:54.956500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5059b4b-7925-4004-a039-37d81c854fde admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:54.967313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1141/2281] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:14:54.971564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:54.981739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecffa26-800f-4f62-ba93-0134f876689d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:54.986685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:54.987349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:54.987630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:54.987950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] PATH_INFO: `/v3/projects/f4e5e9d10a8e4d98b6dfff10362f5e8d/users/c3e88a4234f64c3b95f313a0355f0b81/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:55.004419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e5e9d10a8e4d98b6dfff10362f5e8d, user_id=c3e88a4234f64c3b95f313a0355f0b81, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:55.007147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:55.021450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1141/2282] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:14:54 2022] PUT /identity/v3/projects/f4e5e9d10a8e4d98b6dfff10362f5e8d/users/c3e88a4234f64c3b95f313a0355f0b81/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:14:55.032116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d710-7562-435b-ad7a-f85b929d5e22 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:55.032406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d710-7562-435b-ad7a-f85b929d5e22 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:55.032683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d710-7562-435b-ad7a-f85b929d5e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:55.062101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c152d710-7562-435b-ad7a-f85b929d5e22 None None] MFA Rules not processed for user `c3e88a4234f64c3b95f313a0355f0b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:55.063319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c152d710-7562-435b-ad7a-f85b929d5e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:55.129323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1142/2283] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:14:55.146438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:55.149317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:55.206022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:55.211607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f69c8-7fa6-4616-b12e-9470c4485457 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:55.219889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:55.220529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:55.220827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:55.221112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:55.230125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:55.231558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:55.240658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1142/2284] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:56.652216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14523e7a-d386-40b4-98b2-0ca5289ec4ab None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:56.652774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14523e7a-d386-40b4-98b2-0ca5289ec4ab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:56.653027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14523e7a-d386-40b4-98b2-0ca5289ec4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:56.681052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-14523e7a-d386-40b4-98b2-0ca5289ec4ab None None] MFA Rules not processed for user `c3e88a4234f64c3b95f313a0355f0b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:56.682302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-14523e7a-d386-40b4-98b2-0ca5289ec4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:56.689296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1143/2285] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:58.497375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:58.499207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:58.573078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790752f8-2500-464f-9dfe-c5dba43aa22f service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:58.577768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:58.578409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:58.578675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:58.578949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] PATH_INFO: `/v3/users/2d198fe34157430cba464fe85954872b` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:58.580157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d198fe34157430cba464fe85954872b)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:58.582085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:58.628815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d198fe34157430cba464fe85954872b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:58.629482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d198fe34157430cba464fe85954872b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:58.636318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d198fe34157430cba464fe85954872b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:58.636959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d198fe34157430cba464fe85954872b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:58.646473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d198fe34157430cba464fe85954872b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:58.667542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1143/2286] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:14:58 2022] DELETE /identity/v3/users/2d198fe34157430cba464fe85954872b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:14:58.672237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:58.672703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:58.672815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:58.691934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:14:58.693138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:58.751777 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1144/2287] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:14:58.761849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:58.763550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:58.830811 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dcb05-a7c4-4028-b0be-6b0dbaf4de53 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:58.834572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:58.835218 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:58.835494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:58.835781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:58.844813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:58.846332 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:58.855862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1144/2288] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:14:59.141589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:14:59.143309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:14:59.157451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a5a50-e80f-4bea-8aaf-132a0d70b7e6 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:14:59.164173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:14:59.165425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:14:59.165846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:14:59.166259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] PATH_INFO: `/v3/projects/83b28a5887a84cd395282029952f0b25` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:14:59.168554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b28a5887a84cd395282029952f0b25)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:14:59.170570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:14:59.218424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b28a5887a84cd395282029952f0b25', 'request_id': 'req-bc992886-66be-4618-8817-26330c7e3ac5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:59.223858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b28a5887a84cd395282029952f0b25', 'request_id': 'req-bc992886-66be-4618-8817-26330c7e3ac5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:59.229893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b28a5887a84cd395282029952f0b25', 'request_id': 'req-bc992886-66be-4618-8817-26330c7e3ac5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:59.240544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:14:59.246199 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] The token cache is being invalidate because project 83b28a5887a84cd395282029952f0b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:14:59.246689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:59.247424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bc992886-66be-4618-8817-26330c7e3ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:14:59.249488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1145/2289] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:14:59 2022] DELETE /identity/v3/projects/83b28a5887a84cd395282029952f0b25 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:08.851441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:09.251128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:20.603129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:20.604606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:20.624005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3926c7-bfa2-47eb-bfa9-072328727ac6 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:20.705341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:20.706160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:20.706567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:20.707080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] PATH_INFO: `/v3/users/a5a0ce0c51104920b8804aa19fbd626c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:20.708484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a0ce0c51104920b8804aa19fbd626c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:20.710613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:20.767213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a0ce0c51104920b8804aa19fbd626c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:20.768100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a0ce0c51104920b8804aa19fbd626c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:20.775541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a0ce0c51104920b8804aa19fbd626c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:20.776302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a0ce0c51104920b8804aa19fbd626c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:20.786029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a0ce0c51104920b8804aa19fbd626c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:20.806183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1145/2290] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:20 2022] DELETE /identity/v3/users/a5a0ce0c51104920b8804aa19fbd626c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:20.810963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:20.811542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:20.811542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:20.832733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:20.833960 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:20.895270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1146/2291] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:20.907577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:20.909183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:20.977274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76b650-c42d-49e7-a385-bfa954371c05 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:20.981938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:20.982568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:20.982846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:20.983119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:20.997643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:20.999555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:21.011860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1146/2292] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:21.381985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:21.384167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:21.398681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dda6a-837b-43f4-b4ee-d7cd5ae830dc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:21.404689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:21.405353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:21.405628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:21.405900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] PATH_INFO: `/v3/projects/2a5e23d151c943b78dec682dae9cdc4f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:21.407073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5e23d151c943b78dec682dae9cdc4f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:21.409119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:21.459021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5e23d151c943b78dec682dae9cdc4f', 'request_id': 'req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.464907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5e23d151c943b78dec682dae9cdc4f', 'request_id': 'req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.471655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5e23d151c943b78dec682dae9cdc4f', 'request_id': 'req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.489749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] The token cache is being invalidate because project 2a5e23d151c943b78dec682dae9cdc4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:21.489976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.490738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.492302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1147/2293] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:21 2022] DELETE /identity/v3/projects/2a5e23d151c943b78dec682dae9cdc4f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:21.496277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:21.506037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72f2f-ab45-4ce4-b1b6-fc319d15fadc service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:21.524073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:21.565831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:21.566577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:21.566856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:21.567125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] PATH_INFO: `/v3/users/82c1f2a26ce84806b59ce1171176ee49` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:21.568290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c1f2a26ce84806b59ce1171176ee49)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:21.570230 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:21.619347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c1f2a26ce84806b59ce1171176ee49'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.619980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c1f2a26ce84806b59ce1171176ee49'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.626623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c1f2a26ce84806b59ce1171176ee49'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.627257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c1f2a26ce84806b59ce1171176ee49'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.637892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-379cca5c-8b20-4691-80e6-68093920f1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c1f2a26ce84806b59ce1171176ee49'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:21.655611 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1147/2294] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:21 2022] DELETE /identity/v3/users/82c1f2a26ce84806b59ce1171176ee49 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:21.938292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:21.939826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:21.953310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac4359-93b1-4f00-acbc-2e45e3e6fac3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.018661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.019322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.019629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.019896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] PATH_INFO: `/v3/projects/8cbe14b8b1634119b4e1547a179814ac` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.021076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbe14b8b1634119b4e1547a179814ac)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.023085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.055069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:22.071932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbe14b8b1634119b4e1547a179814ac', 'request_id': 'req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:22.077278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbe14b8b1634119b4e1547a179814ac', 'request_id': 'req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:22.083255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbe14b8b1634119b4e1547a179814ac', 'request_id': 'req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:22.097233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] The token cache is being invalidate because project 8cbe14b8b1634119b4e1547a179814ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:22.097520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:22.098195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cbff9f9d-7aa4-486a-895c-8d73c03f4fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:22.099834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1148/2295] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:21 2022] DELETE /identity/v3/projects/8cbe14b8b1634119b4e1547a179814ac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:22.104771 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.105080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.105361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.125601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:22.126819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:22.200283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1148/2296] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:22.208298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.209010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.209586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.229625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:22.231256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:22.239477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1149/2297] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:22.243674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.254281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb40aa-67b7-4c0f-af4f-00290ec76b63 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.259715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.260449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.260730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.261000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.261815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:15:22.262118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.263848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.279352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1149/2298] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:15:22.280810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.291851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba3185-acdc-467d-b0ba-d3562c31bb74 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.297887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.298655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.298958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.299248 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.300152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.302005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.332715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1150/2299] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:22.336971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.347366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce3bc-22df-476b-961a-1b2ea4ade287 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.375398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:22.411457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.412132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.412407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.412682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.413353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.415782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.438053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1150/2300] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:15:22.442586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.453426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac25d5-50b9-4fd9-9f24-5a2762e6b0ff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.458112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.458735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.459016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.459290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.460017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.461782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.472290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1151/2301] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:22.476328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.486514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776371c-1a5c-42b5-9a52-20b50156fc68 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.491137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464919b-de2f-4470-9ded-5979206ac333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.491759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464919b-de2f-4470-9ded-5979206ac333 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.492071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464919b-de2f-4470-9ded-5979206ac333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.492346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464919b-de2f-4470-9ded-5979206ac333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.493050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464919b-de2f-4470-9ded-5979206ac333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.494649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464919b-de2f-4470-9ded-5979206ac333 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.505678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1151/2302] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:22.510314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.521366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60245-9f2c-4389-a9ac-91e58235c187 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.526771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.527541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.527869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.528137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] PATH_INFO: `/v3/projects/0a01a84170234c99bac07c8bb184d824/users/1c3bce87172a42129220b4502f94823e/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.544376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a01a84170234c99bac07c8bb184d824, user_id=1c3bce87172a42129220b4502f94823e, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.547550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.563688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1152/2303] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:22 2022] PUT /identity/v3/projects/0a01a84170234c99bac07c8bb184d824/users/1c3bce87172a42129220b4502f94823e/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:15:22.575058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.575353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.575634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.606750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] MFA Rules not processed for user `1c3bce87172a42129220b4502f94823e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:22.608002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:22.675438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1152/2304] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:22.682460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.748368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef687-1319-41da-9405-391f5aab8ddc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.753889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.754525 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.754810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.755085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.755759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.757713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.787313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1153/2305] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:22.791824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.802756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5591137-7622-42c4-9864-f9d0c2e80003 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.861976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.862616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.862899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.863175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.863886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.865906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.888863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1153/2306] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:22 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:22.892771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.903738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6fb3c-8297-4574-b5e4-5691fb1b25f8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.908668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.909375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.909682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.909959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.910660 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.912413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.924239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1154/2307] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:22.928410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.938567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6870e0-de0a-4c85-9985-d27642d49523 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:22.943350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:22.944067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:22.944366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:22.944631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] PATH_INFO: `/v3/projects/e817ce97fd76477caf95ae8e366e8c41/users/659fce286fda4b268f950b71b536b2ab/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:22.959635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e817ce97fd76477caf95ae8e366e8c41, user_id=659fce286fda4b268f950b71b536b2ab, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:22.962330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:22.976738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1154/2308] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:22 2022] PUT /identity/v3/projects/e817ce97fd76477caf95ae8e366e8c41/users/659fce286fda4b268f950b71b536b2ab/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:22.980983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:22.992930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3604dff-fad0-4387-ab05-7bf6ad9ac553 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:23.021700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:23.062320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccfcad-cce7-4514-b071-b1699d9ffd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:23.062952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccfcad-cce7-4514-b071-b1699d9ffd7f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.063235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccfcad-cce7-4514-b071-b1699d9ffd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.063509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccfcad-cce7-4514-b071-b1699d9ffd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.064244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccfcad-cce7-4514-b071-b1699d9ffd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:23.065958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccfcad-cce7-4514-b071-b1699d9ffd7f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:23.076958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1155/2309] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:23.081213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:23.086781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:23.092820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1fa58-50e4-4e6d-b155-fcb250bc67ad admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:23.099933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:23.100619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.100907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.101183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] PATH_INFO: `/v3/projects/e817ce97fd76477caf95ae8e366e8c41/users/659fce286fda4b268f950b71b536b2ab/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.103382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e817ce97fd76477caf95ae8e366e8c41, user_id=659fce286fda4b268f950b71b536b2ab, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:23.106232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:23.117671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1155/2310] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:23 2022] PUT /identity/v3/projects/e817ce97fd76477caf95ae8e366e8c41/users/659fce286fda4b268f950b71b536b2ab/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:23.128375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.128664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.128942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.156394 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] MFA Rules not processed for user `659fce286fda4b268f950b71b536b2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:23.157606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:23.233566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1156/2311] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:23.245145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:23.247075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:23.316698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639a295-31dc-472c-9d81-6123563712ea admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:23.320636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80807a-a20a-4cb2-b349-37427c86a8b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:23.321262 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80807a-a20a-4cb2-b349-37427c86a8b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.321626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80807a-a20a-4cb2-b349-37427c86a8b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.321933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80807a-a20a-4cb2-b349-37427c86a8b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.333114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80807a-a20a-4cb2-b349-37427c86a8b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:23.334579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80807a-a20a-4cb2-b349-37427c86a8b7 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:23.346460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1156/2312] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:23.420300 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:23.494231 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a3243-763b-429d-8e85-f3a8675dc9cc None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:23.500325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:23.501077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.501455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.501762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] PATH_INFO: `/v3/users/1c3bce87172a42129220b4502f94823e` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.508029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3bce87172a42129220b4502f94823e)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:23.511018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:23.569654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3bce87172a42129220b4502f94823e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:23.570374 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3bce87172a42129220b4502f94823e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:23.572047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:23.582902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3bce87172a42129220b4502f94823e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:23.583728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3bce87172a42129220b4502f94823e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:23.596483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3bce87172a42129220b4502f94823e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:23.629247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1157/2313] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:23 2022] DELETE /identity/v3/users/1c3bce87172a42129220b4502f94823e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:23.630139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.630518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.630886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.660882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:23.662124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:23.732705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1157/2314] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:23.743317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:23.745139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:23.810441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8c8b8-409a-4667-9386-992c17a5a50f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:23.814841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:23.815486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:23.815785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:23.816086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:23.827992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:23.829626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:23.840328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1158/2315] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:24.008627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:24.010364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.027672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc26b96-e9a1-49c6-88bd-1810653b98e1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:24.034590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:24.035537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.036121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.036486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] PATH_INFO: `/v3/users/527be219ee9e44c18f2e0f24e129a76a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.037873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=527be219ee9e44c18f2e0f24e129a76a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:24.040282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:24.105768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '527be219ee9e44c18f2e0f24e129a76a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.106886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '527be219ee9e44c18f2e0f24e129a76a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.115113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:24.116590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.117160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '527be219ee9e44c18f2e0f24e129a76a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.117853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '527be219ee9e44c18f2e0f24e129a76a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.132630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4bc2f2b0-54f1-4d5f-a9c3-8620fc7bc16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '527be219ee9e44c18f2e0f24e129a76a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.159284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1158/2316] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:24 2022] DELETE /identity/v3/users/527be219ee9e44c18f2e0f24e129a76a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:24.161091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acaa9f-f31f-4dcf-9b3e-2fc1b4748ec8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:24.165741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.166096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.166438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.176424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:24.192280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:24.193813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.229207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:24.229893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.230171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.230598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] PATH_INFO: `/v3/projects/0a01a84170234c99bac07c8bb184d824` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.231866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a01a84170234c99bac07c8bb184d824)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:24.233874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:24.268004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1159/2317] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:24.279238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:24.282900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.299220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a01a84170234c99bac07c8bb184d824', 'request_id': 'req-78a94c67-5115-4c42-b205-116e66c19609'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.307606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a01a84170234c99bac07c8bb184d824', 'request_id': 'req-78a94c67-5115-4c42-b205-116e66c19609'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.314949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a01a84170234c99bac07c8bb184d824', 'request_id': 'req-78a94c67-5115-4c42-b205-116e66c19609'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.329359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:24.341869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] The token cache is being invalidate because project 0a01a84170234c99bac07c8bb184d824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:24.342170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.343379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.345357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1159/2318] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:24 2022] DELETE /identity/v3/projects/0a01a84170234c99bac07c8bb184d824 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:15:24.349342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:24.351721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.367566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4c647-7d91-4940-8a6f-d795d76274e6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:24.467598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:24.468666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.469061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.469335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.472270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94c67-5115-4c42-b205-116e66c19609 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:24.475266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.482213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:24.483333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.483604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.483921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] PATH_INFO: `/v3/users/659fce286fda4b268f950b71b536b2ab` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.485622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] RBAC: Authorizing `identity:delete_user(user_id=659fce286fda4b268f950b71b536b2ab)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:24.488288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:24.503014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:24.504694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:24.521081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1160/2319] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:15:24.549179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659fce286fda4b268f950b71b536b2ab'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.549994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659fce286fda4b268f950b71b536b2ab'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.557520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659fce286fda4b268f950b71b536b2ab'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.558249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659fce286fda4b268f950b71b536b2ab'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.569271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659fce286fda4b268f950b71b536b2ab'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:24.594692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1160/2320] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:24 2022] DELETE /identity/v3/users/659fce286fda4b268f950b71b536b2ab => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:15:24.887535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:24.889276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.904801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb72d9-912f-4505-8da1-0c9a418921b4 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:24.959402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:24.960816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:24.963817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:24.964582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.964999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.965339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] PATH_INFO: `/v3/projects/37a44af0e30841049a0c40b42790dba8` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.966639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a44af0e30841049a0c40b42790dba8)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:24.969462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:24.977977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:24.983281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3f94-6f82-4ebf-960c-d911124a521a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:24.988156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:24.988810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:24.989091 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:24.989365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] PATH_INFO: `/v3/projects/e817ce97fd76477caf95ae8e366e8c41` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:24.990658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e817ce97fd76477caf95ae8e366e8c41)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:24.997657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.029353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a44af0e30841049a0c40b42790dba8', 'request_id': 'req-bdaeff81-d8c5-436d-980d-73ca8d3211b6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.034778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a44af0e30841049a0c40b42790dba8', 'request_id': 'req-bdaeff81-d8c5-436d-980d-73ca8d3211b6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.041333 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a44af0e30841049a0c40b42790dba8', 'request_id': 'req-bdaeff81-d8c5-436d-980d-73ca8d3211b6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.043837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e817ce97fd76477caf95ae8e366e8c41', 'request_id': 'req-e3e9bf9a-97e6-49f9-9542-07164c2440cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.050889 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e817ce97fd76477caf95ae8e366e8c41', 'request_id': 'req-e3e9bf9a-97e6-49f9-9542-07164c2440cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.054536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] The token cache is being invalidate because project 37a44af0e30841049a0c40b42790dba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:25.054824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.055431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-bdaeff81-d8c5-436d-980d-73ca8d3211b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.056987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1161/2321] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:24 2022] DELETE /identity/v3/projects/37a44af0e30841049a0c40b42790dba8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:25.061554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e817ce97fd76477caf95ae8e366e8c41', 'request_id': 'req-e3e9bf9a-97e6-49f9-9542-07164c2440cb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.062010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeec04-6859-42c1-9eee-8dcb3348034f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.062319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeec04-6859-42c1-9eee-8dcb3348034f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.062598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeec04-6859-42c1-9eee-8dcb3348034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.071008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:25.075507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] The token cache is being invalidate because project e817ce97fd76477caf95ae8e366e8c41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:25.075967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.077175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e3e9bf9a-97e6-49f9-9542-07164c2440cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:25.078910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1161/2322] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:24 2022] DELETE /identity/v3/projects/e817ce97fd76477caf95ae8e366e8c41 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:25.082020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-01eeec04-6859-42c1-9eee-8dcb3348034f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:25.083294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-01eeec04-6859-42c1-9eee-8dcb3348034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.084642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c822d7-1ad0-449a-9224-969e3136e55a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.085210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c822d7-1ad0-449a-9224-969e3136e55a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.086345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c822d7-1ad0-449a-9224-969e3136e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.107654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-50c822d7-1ad0-449a-9224-969e3136e55a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:25.108900 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-50c822d7-1ad0-449a-9224-969e3136e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.144477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1162/2323] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:25.153438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.153727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.154007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.167903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1162/2324] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:25.175725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.176049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.176331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.177025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:25.178139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.186972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1163/2325] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:25.190937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.198603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:25.200114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.201015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518532c4-e0a1-4806-b5fb-dff699d7c3b0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.205625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.206235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.206529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.206780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.207511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:15:25.207863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.209455 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.209936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1163/2326] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:25.214055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.222103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1164/2327] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:15:25.223877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f6def-dc8d-46ff-8b87-0a933268bf6f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.226161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.228926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.229632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.229916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.230190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.231157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:15:25.231477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.233712 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.235878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba1d1f-592b-4636-aa2e-e5b5d16134b7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.240556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.241174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.241452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.241724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.242392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.244300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.245932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1164/2328] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 05:15:25.250833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.262079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a668271-eb36-4293-b46a-0fc90834d7ea admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.268044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.268800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.269235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.269608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.270383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.272458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.276381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1165/2329] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:15:25.280780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.314759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1165/2330] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:15:25.316652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.338506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:25.348825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77282b2-45e5-4b41-b861-c82f584309c8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.395107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd45508-cd85-472b-9a76-34d85721a201 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.401377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.402164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.402498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.402892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.403672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.405791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.415719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.416525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.416879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.417223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.418073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.420114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.429023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1166/2331] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:25.433431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.443503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768521b0-6c72-4027-b33c-88b328488f40 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.444006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1166/2332] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:15:25.447751 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.449903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.450552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.450857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.451133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.451894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.453546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.457544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d63b1-1936-44bd-8076-e16f344c8077 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.462616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.463232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.463552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.463817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.464112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1167/2333] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:15:25.467448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.468338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.469126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.480913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1167/2334] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:15:25.481285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300aebb2-d641-4fe6-b833-2de4997afbf3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.485134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.488556 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.489836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.490142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.490535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.491262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.493112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.497479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585443f-f4ce-48ab-8372-e170ec26a92b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.505335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.506063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.506829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.506829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.506995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1168/2335] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:25.508442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.510247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.512685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.523298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1168/2336] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:25.523478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82cd1a-fff9-47e4-929b-83bf75b53a34 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.527429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.528273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74475a22-4ba4-476e-adbf-144db2b120dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.528952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74475a22-4ba4-476e-adbf-144db2b120dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.529238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74475a22-4ba4-476e-adbf-144db2b120dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.529520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74475a22-4ba4-476e-adbf-144db2b120dc admin admin] PATH_INFO: `/v3/projects/c3a9b7d7692342fcb1445ff20d9043cd/users/8a85da0b76864346af3003872ce1b1d1/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.537287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cfb0e-3eb1-4a94-8002-ef1e0bec6517 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.544163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220bcbb-e397-4751-a58e-3171ac246bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.544848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcbb-e397-4751-a58e-3171ac246bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.545134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcbb-e397-4751-a58e-3171ac246bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.545414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcbb-e397-4751-a58e-3171ac246bb3 admin admin] PATH_INFO: `/v3/projects/d261590b6bb44db985accae6b8b52eb6/users/71f5b9ca081e46bc8046e4114d23332c/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.550358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74475a22-4ba4-476e-adbf-144db2b120dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a9b7d7692342fcb1445ff20d9043cd, user_id=8a85da0b76864346af3003872ce1b1d1, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.553709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74475a22-4ba4-476e-adbf-144db2b120dc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.563383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcbb-e397-4751-a58e-3171ac246bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d261590b6bb44db985accae6b8b52eb6, user_id=71f5b9ca081e46bc8046e4114d23332c, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.566330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcbb-e397-4751-a58e-3171ac246bb3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.572226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1169/2337] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:25 2022] PUT /identity/v3/projects/c3a9b7d7692342fcb1445ff20d9043cd/users/8a85da0b76864346af3003872ce1b1d1/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:25.584931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.585583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.585782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.596401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:25.615825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1169/2338] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:25 2022] PUT /identity/v3/projects/d261590b6bb44db985accae6b8b52eb6/users/71f5b9ca081e46bc8046e4114d23332c/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:15:25.618741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.619033 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.619327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.626673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] MFA Rules not processed for user `8a85da0b76864346af3003872ce1b1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:25.628046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.647377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] MFA Rules not processed for user `71f5b9ca081e46bc8046e4114d23332c`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:25.648626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.707042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1170/2339] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:25.710601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.725650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1170/2340] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:25.737940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.739574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:25.782048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:25.811029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbd3f3-8a18-4014-815d-cda50242a6d7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.815038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.815184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac1d9-5fe5-4117-81f3-318ccbfc7860 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.815749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.816154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.816498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.820315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.820936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.821223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.821516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.822193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.824318 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.827043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.833353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.844193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1171/2341] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:15:25.854041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1171/2342] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:25.858067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.868628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31370d06-31c9-4445-91f6-a697e1b589af service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.925259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.925947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.926235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.926517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.927217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.929190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.954897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1172/2343] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:25 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:25.956376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:25.967009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81ba5-41d0-4582-bf43-44f302a784d4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:25.971934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:25.972557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:25.972841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:25.973128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:25.973812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:25.975424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:25.989426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1172/2344] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:15:25.990770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:26.001330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e82a11-3a2c-44d2-bb41-d3f94affa243 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:26.006193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:26.006883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:26.007208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:26.007504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] PATH_INFO: `/v3/projects/902b1263a30b4d719c9540dd520d4f6b/users/6f9194186a1043ef8f3525267f961e69/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:26.038898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=902b1263a30b4d719c9540dd520d4f6b, user_id=6f9194186a1043ef8f3525267f961e69, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:26.042119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:26.060003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1173/2345] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:25 2022] PUT /identity/v3/projects/902b1263a30b4d719c9540dd520d4f6b/users/6f9194186a1043ef8f3525267f961e69/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:26.064082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:26.075530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55ca6-5270-4fe5-b9f4-1d69ecdb8134 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:26.135898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ea075-82c6-42ab-a941-784bddfe75af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:26.136536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ea075-82c6-42ab-a941-784bddfe75af admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:26.136894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ea075-82c6-42ab-a941-784bddfe75af admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:26.137163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ea075-82c6-42ab-a941-784bddfe75af admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:26.138041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ea075-82c6-42ab-a941-784bddfe75af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:26.139973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ea075-82c6-42ab-a941-784bddfe75af admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:26.151135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1173/2346] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:26.155693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:26.167793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d5f4b-692d-4d28-a231-42e0fe8ea38f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:26.173176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:26.173880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:26.174146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:26.174418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] PATH_INFO: `/v3/projects/902b1263a30b4d719c9540dd520d4f6b/users/6f9194186a1043ef8f3525267f961e69/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:26.176497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=902b1263a30b4d719c9540dd520d4f6b, user_id=6f9194186a1043ef8f3525267f961e69, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:26.179146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:26.190821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1174/2347] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:26 2022] PUT /identity/v3/projects/902b1263a30b4d719c9540dd520d4f6b/users/6f9194186a1043ef8f3525267f961e69/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:26.201706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:26.202216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:26.202420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:26.210657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:26.236592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] MFA Rules not processed for user `6f9194186a1043ef8f3525267f961e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:26.238085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:26.308277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1174/2348] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:26.319576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:26.321302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:26.384901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185dd6c-85b1-40cc-b88c-d7c8c57d7b73 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:26.389167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b02f7-f7a4-49a6-b604-4da43d7c4753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:26.389750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b02f7-f7a4-49a6-b604-4da43d7c4753 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:26.390028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b02f7-f7a4-49a6-b604-4da43d7c4753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:26.390307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b02f7-f7a4-49a6-b604-4da43d7c4753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:26.401613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b02f7-f7a4-49a6-b604-4da43d7c4753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:26.403278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b02f7-f7a4-49a6-b604-4da43d7c4753 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:26.413702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1175/2349] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:15:27.021590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:27.084214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8f09-c75e-49ea-9b8f-39f25f9081a5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:27.089129 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:27.089786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:27.090108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:27.090384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] PATH_INFO: `/v3/users/8a85da0b76864346af3003872ce1b1d1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:27.091575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a85da0b76864346af3003872ce1b1d1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:27.093557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:27.142434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a85da0b76864346af3003872ce1b1d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.143078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a85da0b76864346af3003872ce1b1d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.149818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a85da0b76864346af3003872ce1b1d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.150439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a85da0b76864346af3003872ce1b1d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.159663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a85da0b76864346af3003872ce1b1d1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.176833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1175/2350] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:27 2022] DELETE /identity/v3/users/8a85da0b76864346af3003872ce1b1d1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:27.181540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:27.182019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:27.182111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:27.202187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:27.203411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:27.230015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:27.266364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1176/2351] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:27.276981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:27.278468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:27.289549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:27.345583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26a8d-9ef1-4962-9db2-0532a62fc930 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:27.345583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:27.345946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:27.346256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:27.346531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:27.355790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:27.357243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:27.367108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1176/2352] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:27.697432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:27.698884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:27.713458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de7310-9dcd-4d80-a0f6-e04dfdd61bf3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:27.719782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:27.722066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:27.722583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:27.723111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] PATH_INFO: `/v3/projects/c3a9b7d7692342fcb1445ff20d9043cd` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:27.724626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a9b7d7692342fcb1445ff20d9043cd)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:27.727212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:27.777306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a9b7d7692342fcb1445ff20d9043cd', 'request_id': 'req-869f3f22-0a15-4647-99f5-1f89c24fafe7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.782735 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a9b7d7692342fcb1445ff20d9043cd', 'request_id': 'req-869f3f22-0a15-4647-99f5-1f89c24fafe7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.789563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a9b7d7692342fcb1445ff20d9043cd', 'request_id': 'req-869f3f22-0a15-4647-99f5-1f89c24fafe7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.803191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] The token cache is being invalidate because project c3a9b7d7692342fcb1445ff20d9043cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:27.803479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.804104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-869f3f22-0a15-4647-99f5-1f89c24fafe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.805617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1177/2353] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:27 2022] DELETE /identity/v3/projects/c3a9b7d7692342fcb1445ff20d9043cd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:27.810478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:27.820791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739092d9-6f7c-4d30-ba4c-7978cebbbcd2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:27.883322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:27.884009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:27.884292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:27.884723 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] PATH_INFO: `/v3/users/6f9194186a1043ef8f3525267f961e69` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:27.885721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9194186a1043ef8f3525267f961e69)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:27.887737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:27.937311 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9194186a1043ef8f3525267f961e69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.937943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9194186a1043ef8f3525267f961e69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.944875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9194186a1043ef8f3525267f961e69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.945485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9194186a1043ef8f3525267f961e69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.954405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9194186a1043ef8f3525267f961e69'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:27.972772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1177/2354] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:27 2022] DELETE /identity/v3/users/6f9194186a1043ef8f3525267f961e69 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:28.143086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.143386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.143667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.162953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:28.164577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:28.236124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1178/2355] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:15:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:28.278839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.280518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:28.308665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.310318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:28.319167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:28.355418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90f751-5166-4074-a29a-d6e0e051135a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.360162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.360809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.361095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.361369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] PATH_INFO: `/v3/projects/902b1263a30b4d719c9540dd520d4f6b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.362595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] RBAC: Authorizing `identity:delete_project(project_id=902b1263a30b4d719c9540dd520d4f6b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.364603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.376844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:28.383640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b66d6-6f4c-478c-8b2f-a41e2284844f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.390198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11365e-9eef-4123-9722-68abbc16b24b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.390866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11365e-9eef-4123-9722-68abbc16b24b service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.391212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11365e-9eef-4123-9722-68abbc16b24b service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.391540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11365e-9eef-4123-9722-68abbc16b24b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.400538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11365e-9eef-4123-9722-68abbc16b24b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.402116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11365e-9eef-4123-9722-68abbc16b24b service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.410126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1179/2356] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:15:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:15:28.414551 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902b1263a30b4d719c9540dd520d4f6b', 'request_id': 'req-ad6d8913-f2eb-494c-af86-80797d86babd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:28.420454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902b1263a30b4d719c9540dd520d4f6b', 'request_id': 'req-ad6d8913-f2eb-494c-af86-80797d86babd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:28.426932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902b1263a30b4d719c9540dd520d4f6b', 'request_id': 'req-ad6d8913-f2eb-494c-af86-80797d86babd'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:28.441313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] The token cache is being invalidate because project 902b1263a30b4d719c9540dd520d4f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:28.441622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:28.442227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ad6d8913-f2eb-494c-af86-80797d86babd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:28.443867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1178/2357] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:28 2022] DELETE /identity/v3/projects/902b1263a30b4d719c9540dd520d4f6b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:28.449151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.449527 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.449921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.469475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:28.470665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:28.537415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1180/2358] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:28.546507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.546795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.547072 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.570073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:28.571544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:28.580098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1179/2359] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:28.584780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.595280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531071c-ff8f-40d0-897e-7f9c637d4972 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.600086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501a059-16c8-4262-9844-355375194760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.600702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501a059-16c8-4262-9844-355375194760 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.600979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501a059-16c8-4262-9844-355375194760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.601252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501a059-16c8-4262-9844-355375194760 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.601946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501a059-16c8-4262-9844-355375194760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:15:28.602248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501a059-16c8-4262-9844-355375194760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.603866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501a059-16c8-4262-9844-355375194760 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.616663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1181/2360] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:15:28.621016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.630773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8c5d5-9602-4b43-a9b1-75f4c94ed57e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.636508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.637223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.637506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.637780 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.638721 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.641298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.680501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1180/2361] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:28 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:28.684764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501a059-16c8-4262-9844-355375194760 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.696434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501a059-16c8-4262-9844-355375194760 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.780518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.781170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.781454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.781736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.782557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.788662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.827650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1182/2362] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:28 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:28.833288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.860399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bfb77-dc91-4b67-9d42-a52c62843d77 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.867342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.868077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.868524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.868807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.869915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.872316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.886206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1181/2363] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:28.890379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.902837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92849f-04f1-4179-8463-15f592bd8c09 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.908264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9dcb4-4900-40a6-bc10-3113069becde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.909102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9dcb4-4900-40a6-bc10-3113069becde admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.909476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9dcb4-4900-40a6-bc10-3113069becde admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.909843 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9dcb4-4900-40a6-bc10-3113069becde admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:28.910736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9dcb4-4900-40a6-bc10-3113069becde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:28.912733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9dcb4-4900-40a6-bc10-3113069becde admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:28.940621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1183/2364] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:28.945077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:28.973825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30025a-981c-41a7-b754-7460da69d479 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:28.984391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78db1b6-c323-44a0-b321-0949be89a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:28.985180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db1b6-c323-44a0-b321-0949be89a94e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:28.985473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db1b6-c323-44a0-b321-0949be89a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:28.985862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db1b6-c323-44a0-b321-0949be89a94e admin admin] PATH_INFO: `/v3/projects/3f2ac2a2211841b5a156d98fa3c5868e/users/5526932cb97242a6808759cf715f14c4/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.008902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db1b6-c323-44a0-b321-0949be89a94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2ac2a2211841b5a156d98fa3c5868e, user_id=5526932cb97242a6808759cf715f14c4, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:29.012626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db1b6-c323-44a0-b321-0949be89a94e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:29.052834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1182/2365] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:28 2022] PUT /identity/v3/projects/3f2ac2a2211841b5a156d98fa3c5868e/users/5526932cb97242a6808759cf715f14c4/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:29.063861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:29.063861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:29.063861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.070229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:29.072335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:29.079371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.111435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:29.113667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:29.138328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] MFA Rules not processed for user `5526932cb97242a6808759cf715f14c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:29.139509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:29.213564 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:29.216231 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:29.228805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1184/2366] 10.222.0.30 () {58 vars in 967 bytes} [Fri Aug 5 05:15:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:29.243904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:29.253714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1183/2367] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:29.261534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:29.267262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:29.270847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:29.357421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c13c9b-0bc6-4dce-ba3c-0bb0f25e3c44 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:29.362051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccd4ed-fc69-4f4b-ac9a-cae435aec972 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:29.363303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded49e55-7dc4-45c2-adf3-bbf1aded21c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:29.363987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded49e55-7dc4-45c2-adf3-bbf1aded21c6 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:29.364234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded49e55-7dc4-45c2-adf3-bbf1aded21c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:29.364560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded49e55-7dc4-45c2-adf3-bbf1aded21c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.367866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:29.368766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:29.369391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:29.369984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.374247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded49e55-7dc4-45c2-adf3-bbf1aded21c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:29.375861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded49e55-7dc4-45c2-adf3-bbf1aded21c6 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:29.391699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:29.391699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:29.402587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1185/2368] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:15:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:15:29.403364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1184/2369] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:15:29.738532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1be21-df5c-4326-9c50-a3767a1f70a4 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:29.738847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1be21-df5c-4326-9c50-a3767a1f70a4 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:29.739179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1be21-df5c-4326-9c50-a3767a1f70a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.778072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-78c1be21-df5c-4326-9c50-a3767a1f70a4 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:29.782937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-78c1be21-df5c-4326-9c50-a3767a1f70a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:29.797499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1186/2370] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:15:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:29.807221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:29.824722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2d77-8629-47c1-93eb-b91a2cedc1ee service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:29.832841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:29.833573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:29.833914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:29.834250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:29.847560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:29.850993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:29.861965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1185/2371] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:15:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:15:30.290079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:30.290555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:30.290649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:30.309409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:30.310616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:30.318946 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1187/2372] 10.222.0.30 () {58 vars in 967 bytes} [Fri Aug 5 05:15:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:30.335398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:30.346031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a70d5c-e251-4b6b-8c33-a5e21624525b service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:30.351358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea4dad-7f81-486b-b2a8-c824d4cb976e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:30.352054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea4dad-7f81-486b-b2a8-c824d4cb976e service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:30.352342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea4dad-7f81-486b-b2a8-c824d4cb976e service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:30.352620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea4dad-7f81-486b-b2a8-c824d4cb976e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:30.361759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea4dad-7f81-486b-b2a8-c824d4cb976e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:30.363349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea4dad-7f81-486b-b2a8-c824d4cb976e service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:30.371881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1186/2373] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:15:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 05:15:32.671717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:32.673791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:32.750428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fefd4-17c1-46c7-8341-ee84e191468a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:32.754904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:32.755552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:32.755866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:32.756142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] PATH_INFO: `/v3/users/c3e88a4234f64c3b95f313a0355f0b81` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:32.757363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e88a4234f64c3b95f313a0355f0b81)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:32.759278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:32.808304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e88a4234f64c3b95f313a0355f0b81'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:32.808968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e88a4234f64c3b95f313a0355f0b81'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:32.816284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e88a4234f64c3b95f313a0355f0b81'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:32.816960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e88a4234f64c3b95f313a0355f0b81'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:32.827451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e88a4234f64c3b95f313a0355f0b81'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:32.839279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:32.846524 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1188/2374] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:32 2022] DELETE /identity/v3/users/c3e88a4234f64c3b95f313a0355f0b81 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:32.851202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:32.851711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:32.851829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:32.870351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:32.871604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:32.938050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1187/2375] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:32.948371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:32.949755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:33.029582 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204305d8-392b-4586-980a-d681868625b6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:33.034517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:33.035164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:33.035472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:33.035728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:33.047401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:33.049704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:33.062109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1189/2376] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:33.256410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:33.257500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:33.292334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a7974-2d72-4754-b420-14a9474180e5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:33.301493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:33.302481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:33.307312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:33.308013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] PATH_INFO: `/v3/projects/f4e5e9d10a8e4d98b6dfff10362f5e8d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:33.309376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e5e9d10a8e4d98b6dfff10362f5e8d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:33.311716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:33.371949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e5e9d10a8e4d98b6dfff10362f5e8d', 'request_id': 'req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:33.377266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:33.378413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e5e9d10a8e4d98b6dfff10362f5e8d', 'request_id': 'req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:33.400067 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e5e9d10a8e4d98b6dfff10362f5e8d', 'request_id': 'req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:33.424667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] The token cache is being invalidate because project f4e5e9d10a8e4d98b6dfff10362f5e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:33.424971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:33.426082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bd975d0f-7f9a-42bf-b4b2-75d9d7fcb7f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:33.427882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1188/2377] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:33 2022] DELETE /identity/v3/projects/f4e5e9d10a8e4d98b6dfff10362f5e8d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:34.426878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:34.428408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:34.441668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9281aed-80a2-472e-9032-ce8eb0041037 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:34.500854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:34.501505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:34.501785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:34.502056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] PATH_INFO: `/v3/users/e8e9fb82330749b582ac4522c4b8695f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:34.503849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e9fb82330749b582ac4522c4b8695f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:34.505987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:34.557142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e9fb82330749b582ac4522c4b8695f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:34.557993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e9fb82330749b582ac4522c4b8695f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:34.569288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e9fb82330749b582ac4522c4b8695f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:34.570022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e9fb82330749b582ac4522c4b8695f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:34.578683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e9fb82330749b582ac4522c4b8695f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:34.600637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1190/2378] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:34 2022] DELETE /identity/v3/users/e8e9fb82330749b582ac4522c4b8695f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:34.605117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:34.605496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:34.605674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:34.629746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:34.631156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:34.699999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1189/2379] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:34.711341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:34.712986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:34.736369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:34.791960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37810063-57ac-4bb3-a75e-0128f6e212b6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:34.791960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56cfd-53b8-4d54-9a30-0b1321993870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:34.791960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56cfd-53b8-4d54-9a30-0b1321993870 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:34.791960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56cfd-53b8-4d54-9a30-0b1321993870 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:34.792516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56cfd-53b8-4d54-9a30-0b1321993870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:34.802994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56cfd-53b8-4d54-9a30-0b1321993870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:34.804511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56cfd-53b8-4d54-9a30-0b1321993870 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:34.814663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1191/2380] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:35.178574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.180676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:35.195559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beccb6-10c2-45a9-b5fe-2fe23acc582d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.205530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.206259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.206713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.207019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] PATH_INFO: `/v3/projects/590cd9c83a6749cc8b9f36a4972a59b0` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.209487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=590cd9c83a6749cc8b9f36a4972a59b0)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.213813 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.274163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '590cd9c83a6749cc8b9f36a4972a59b0', 'request_id': 'req-095d7013-c12d-469e-8982-cb9bf6172abc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:35.281027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '590cd9c83a6749cc8b9f36a4972a59b0', 'request_id': 'req-095d7013-c12d-469e-8982-cb9bf6172abc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:35.289516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '590cd9c83a6749cc8b9f36a4972a59b0', 'request_id': 'req-095d7013-c12d-469e-8982-cb9bf6172abc'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:35.303678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] The token cache is being invalidate because project 590cd9c83a6749cc8b9f36a4972a59b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:35.303934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:35.304609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-095d7013-c12d-469e-8982-cb9bf6172abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:35.306131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1190/2381] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:35 2022] DELETE /identity/v3/projects/590cd9c83a6749cc8b9f36a4972a59b0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:35.311091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.311379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.311658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.331638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:35.332911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:35.415478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1192/2382] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:35.423818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.424203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.424506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.445466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:35.446781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:35.454741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1191/2383] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:35.459112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.469239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea52e4-4e81-4a7c-bd13-c2ba900d7c71 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.473963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.474683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.474969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.475247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.475987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:15:35.476356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.478076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.491813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1193/2384] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:15:35.496200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.507221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98d9c8-b071-4e08-a450-5351401ae240 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.512476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.513158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.513492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.513766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.514500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.516441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.544203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1192/2385] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:35 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:35.548809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.563456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78092f-bd82-4852-a82e-18e1e98ec060 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.624726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.625384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.625668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.625948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.626642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.628732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.653970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1194/2386] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:35 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:35.658237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.668996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203014e0-61c6-4dd0-ae2f-dbc794912d8b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.673763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.674373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.674658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.674939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.675663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.677341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.688167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1193/2387] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:35.692568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.704687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ef19c-3e6d-40a4-9e96-71ac8b0142e2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.709499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea26c1-ca51-4049-90a2-3e5a0119f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.710130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea26c1-ca51-4049-90a2-3e5a0119f5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.710417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea26c1-ca51-4049-90a2-3e5a0119f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.710692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea26c1-ca51-4049-90a2-3e5a0119f5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.711465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea26c1-ca51-4049-90a2-3e5a0119f5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.713273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea26c1-ca51-4049-90a2-3e5a0119f5ec admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.728529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1195/2388] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:15:35.730029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.740426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3829ff1-bfd8-43de-a39d-d0efa70117bb admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:35.745587 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:35.746305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.746585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.746859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] PATH_INFO: `/v3/projects/66f85dd9998446f2a790288b0d139699/users/7381ddf1b3334ec9a75fcdc113a47ae3/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.761738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f85dd9998446f2a790288b0d139699, user_id=7381ddf1b3334ec9a75fcdc113a47ae3, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:35.764791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:35.779751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1194/2389] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:35 2022] PUT /identity/v3/projects/66f85dd9998446f2a790288b0d139699/users/7381ddf1b3334ec9a75fcdc113a47ae3/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:35.791694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:35.792061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:35.792342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:35.825617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] MFA Rules not processed for user `7381ddf1b3334ec9a75fcdc113a47ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:35.827017 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:35.847307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:35.919472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1196/2390] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:35.932264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:35.934218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:35.941206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:36.017945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a49735-90ee-4903-b342-907508914cd8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:36.021757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ad2a5-4eaa-49a9-b154-6a705179c3af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:36.022456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ad2a5-4eaa-49a9-b154-6a705179c3af service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.022806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ad2a5-4eaa-49a9-b154-6a705179c3af service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.023150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ad2a5-4eaa-49a9-b154-6a705179c3af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.034496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ad2a5-4eaa-49a9-b154-6a705179c3af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:36.036163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ad2a5-4eaa-49a9-b154-6a705179c3af service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:36.046651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1195/2391] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:36.071568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:36.137279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48851-58a9-4091-b601-196f16589178 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.137571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48851-58a9-4091-b601-196f16589178 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.137856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48851-58a9-4091-b601-196f16589178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.157657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5cc48851-58a9-4091-b601-196f16589178 None None] MFA Rules not processed for user `c7c57f5e11a84bf98d474837adbb0ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:36.158848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5cc48851-58a9-4091-b601-196f16589178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:36.160426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1435-aef8-42a6-80c7-d03cca5107a5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:36.165334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:36.166049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.166328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.166628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] PATH_INFO: `/v3/users/7381ddf1b3334ec9a75fcdc113a47ae3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.167816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7381ddf1b3334ec9a75fcdc113a47ae3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:36.169762 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:36.224004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7381ddf1b3334ec9a75fcdc113a47ae3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.224747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7381ddf1b3334ec9a75fcdc113a47ae3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.225710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1196/2392] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:36.236216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7381ddf1b3334ec9a75fcdc113a47ae3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.237086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7381ddf1b3334ec9a75fcdc113a47ae3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.249490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7381ddf1b3334ec9a75fcdc113a47ae3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.271292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1197/2393] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:36 2022] DELETE /identity/v3/users/7381ddf1b3334ec9a75fcdc113a47ae3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:36.277101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.277396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.277654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.298201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:36.299669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:36.330979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:36.333637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:36.370003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1197/2394] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:36.382238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:36.388683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:36.393213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:36.444958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f6687-8cd6-4030-8c46-6e0c744427f7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:36.452067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:36.453162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.457029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.458142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] PATH_INFO: `/v3/users/71f5b9ca081e46bc8046e4114d23332c` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.470982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c907-73a6-4e0b-bc89-c89370de47a8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:36.472517 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f5b9ca081e46bc8046e4114d23332c)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:36.476386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:36.478233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b666378-5b54-4991-9a0a-3a7282caa449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:36.478949 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b666378-5b54-4991-9a0a-3a7282caa449 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.479292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b666378-5b54-4991-9a0a-3a7282caa449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.479950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b666378-5b54-4991-9a0a-3a7282caa449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.490867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b666378-5b54-4991-9a0a-3a7282caa449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:36.493036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b666378-5b54-4991-9a0a-3a7282caa449 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:36.499369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:36.512255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1198/2395] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:15:36.556916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f5b9ca081e46bc8046e4114d23332c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.557588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f5b9ca081e46bc8046e4114d23332c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.566093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f5b9ca081e46bc8046e4114d23332c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.566953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f5b9ca081e46bc8046e4114d23332c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.578186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f5b9ca081e46bc8046e4114d23332c'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:36.596900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1198/2396] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:36 2022] DELETE /identity/v3/users/71f5b9ca081e46bc8046e4114d23332c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:36.601661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.601966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.602229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.626401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:36.627758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:36.692741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1199/2397] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:36.705503 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:36.706965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:36.775942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4921e-8aef-4ac5-807b-27ce821dcd94 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:36.781773 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:36.782415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:36.782692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:36.782963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:36.793544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:36.795046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:36.805519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1199/2398] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:37.120965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:37.122620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:37.142976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f760c-e5b9-4ec9-830e-d499f2c97f1f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:37.158694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:37.159702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:37.160473 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:37.160959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] PATH_INFO: `/v3/projects/66f85dd9998446f2a790288b0d139699` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:37.162891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f85dd9998446f2a790288b0d139699)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:37.167165 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:37.249656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f85dd9998446f2a790288b0d139699', 'request_id': 'req-af152d21-44be-4535-8314-cd8a24a39efa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.262555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f85dd9998446f2a790288b0d139699', 'request_id': 'req-af152d21-44be-4535-8314-cd8a24a39efa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.264350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:37.265872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:37.279671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f85dd9998446f2a790288b0d139699', 'request_id': 'req-af152d21-44be-4535-8314-cd8a24a39efa'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.283363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d3bed-d00e-4f2d-9595-28635b25c631 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:37.321356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] The token cache is being invalidate because project 66f85dd9998446f2a790288b0d139699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:37.323620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.324457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.328303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1200/2399] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:37 2022] DELETE /identity/v3/projects/66f85dd9998446f2a790288b0d139699 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:37.394163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:37.406348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:37.406986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:37.407222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:37.407900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] PATH_INFO: `/v3/projects/d261590b6bb44db985accae6b8b52eb6` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:37.409261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d261590b6bb44db985accae6b8b52eb6)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:37.412384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:37.491604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd261590b6bb44db985accae6b8b52eb6', 'request_id': 'req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.498948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd261590b6bb44db985accae6b8b52eb6', 'request_id': 'req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.505984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd261590b6bb44db985accae6b8b52eb6', 'request_id': 'req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.534111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] The token cache is being invalidate because project d261590b6bb44db985accae6b8b52eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:37.534507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.535548 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-04ed5668-f50b-4583-b51b-1e8a5ebb59cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:37.538028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1200/2400] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:37 2022] DELETE /identity/v3/projects/d261590b6bb44db985accae6b8b52eb6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:47.305466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:47.525195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:50.554741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:50.556519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:50.570078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152d21-44be-4535-8314-cd8a24a39efa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:50.628586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:50.629275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:50.629558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:50.629841 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] PATH_INFO: `/v3/users/5526932cb97242a6808759cf715f14c4` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:50.631045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5526932cb97242a6808759cf715f14c4)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:50.633050 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:50.681324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5526932cb97242a6808759cf715f14c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:50.682226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5526932cb97242a6808759cf715f14c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:50.689728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5526932cb97242a6808759cf715f14c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:50.690373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5526932cb97242a6808759cf715f14c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:50.703169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5526932cb97242a6808759cf715f14c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:50.720510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1201/2401] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:50 2022] DELETE /identity/v3/users/5526932cb97242a6808759cf715f14c4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:50.725090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:50.725581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:50.725872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:50.745057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:50.746373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:50.814648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1201/2402] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:15:50.824989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:50.826725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:50.891910 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e721e2-11b8-41e3-b260-6bde862aa39b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:50.895656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961d2ca-78e5-47f2-b1d2-e5af43159a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:50.896395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d2ca-78e5-47f2-b1d2-e5af43159a7d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:50.896679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d2ca-78e5-47f2-b1d2-e5af43159a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:50.896952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d2ca-78e5-47f2-b1d2-e5af43159a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:50.908071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d2ca-78e5-47f2-b1d2-e5af43159a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:50.909537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d2ca-78e5-47f2-b1d2-e5af43159a7d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:50.921676 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1202/2403] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:51.415453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.417043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:51.432912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba087867-e645-4f8b-b172-6c7197ae3d4b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.438438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.439170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.439543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.439938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] PATH_INFO: `/v3/projects/3f2ac2a2211841b5a156d98fa3c5868e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.441243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2ac2a2211841b5a156d98fa3c5868e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.443492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:51.503711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2ac2a2211841b5a156d98fa3c5868e', 'request_id': 'req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:51.510868 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2ac2a2211841b5a156d98fa3c5868e', 'request_id': 'req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:51.517372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2ac2a2211841b5a156d98fa3c5868e', 'request_id': 'req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:51.536409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] The token cache is being invalidate because project 3f2ac2a2211841b5a156d98fa3c5868e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:15:51.536409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:51.537641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c594d091-f6ad-4ffc-bf9a-1b9fc2ed728b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:51.539423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1202/2404] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:15:51 2022] DELETE /identity/v3/projects/3f2ac2a2211841b5a156d98fa3c5868e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:51.544652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.544885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.545178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.566953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:51.568381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:51.590893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:51.639850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1203/2405] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:51.647830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.648708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.648708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.670111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:51.671498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:51.682622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1203/2406] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:51.686992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.697196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885c98-55eb-45ad-970c-261fad93df1f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.702105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.702798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.703078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.703350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.704076 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:15:51.704382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.706030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:51.722275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1204/2407] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:15:51.724123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.734353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c1db-4343-4540-8b15-f865dffc54ea None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.740893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.741474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.741854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.742680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.743110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.745177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:51.780743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1204/2408] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:51 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:51.782250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.793654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93472f17-5717-4498-b296-579bee3e4bec admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.852871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.853488 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.853794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.854068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.854738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.856725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:51.880015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1205/2409] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:51 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:51.884420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.894954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf90ca-a9e6-4fd7-aaff-6aaa2dae4f21 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.900021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.900677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.900965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.901253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.901961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.903586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:51.914358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1205/2410] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:51.918532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.928717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ddfa-3364-4831-9e64-27b092172951 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.933604 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6eb5f-fd28-4b3e-8749-b5570301e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.934221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6eb5f-fd28-4b3e-8749-b5570301e74d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.934510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6eb5f-fd28-4b3e-8749-b5570301e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.934784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6eb5f-fd28-4b3e-8749-b5570301e74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.935479 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6eb5f-fd28-4b3e-8749-b5570301e74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.937163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6eb5f-fd28-4b3e-8749-b5570301e74d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:51.948130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1206/2411] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:51.952479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:51.964004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32404ef4-d67c-4f15-9737-31137b981a06 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:51.969130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:51.969818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:51.970097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:51.970399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] PATH_INFO: `/v3/projects/6c6f05fac48d4e0dbfd9b3b488b756a6/users/202e4f11b5274124aaafdc140fcf5146/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:51.988628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6f05fac48d4e0dbfd9b3b488b756a6, user_id=202e4f11b5274124aaafdc140fcf5146, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:51.991436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.011155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1206/2412] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:51 2022] PUT /identity/v3/projects/6c6f05fac48d4e0dbfd9b3b488b756a6/users/202e4f11b5274124aaafdc140fcf5146/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:52.024520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.024874 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.025158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.056891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] MFA Rules not processed for user `202e4f11b5274124aaafdc140fcf5146`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:52.058267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:52.124633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1207/2413] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:52.129379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.152833 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:52.198607 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30578fc-48b8-4fd4-92a8-a99136801ab2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.203491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.204158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.204441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.204723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.205389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.207571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.240653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1207/2414] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:52 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:15:52.241860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.252741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dc45b-5675-4170-b54e-cbe7999e6702 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.264796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:52.314396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.315035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.315317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.315588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.316399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.318333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.341522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1208/2415] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:52 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:15:52.345797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.356589 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3df115-fda1-42f4-b9db-5011d68f0ac0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.361433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.362069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.362354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.362656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.363439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.365132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.379558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1208/2416] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:52.380705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.395635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97712bb-3b1b-45ee-828c-8db226be03d1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.400361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15009bb-9834-4424-b60b-73652974bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.400976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15009bb-9834-4424-b60b-73652974bd0c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.401254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15009bb-9834-4424-b60b-73652974bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.401656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15009bb-9834-4424-b60b-73652974bd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.402392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15009bb-9834-4424-b60b-73652974bd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.404049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15009bb-9834-4424-b60b-73652974bd0c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.416087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1209/2417] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:52.420763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.432347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45cad0-c5a6-4fbe-a0e2-77b660a0797b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.437310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.437981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.438311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.438585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] PATH_INFO: `/v3/projects/f8c3f3c31e1d4a839a70eef3823e5d41/users/60c198b9027d4ef0a46f4af2fb5acda9/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.454918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c3f3c31e1d4a839a70eef3823e5d41, user_id=60c198b9027d4ef0a46f4af2fb5acda9, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.457693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.472761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1209/2418] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:52 2022] PUT /identity/v3/projects/f8c3f3c31e1d4a839a70eef3823e5d41/users/60c198b9027d4ef0a46f4af2fb5acda9/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:15:52.482840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.483126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.483400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.511432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] MFA Rules not processed for user `60c198b9027d4ef0a46f4af2fb5acda9`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:52.512700 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:52.576596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1210/2419] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:52.581461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.649982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a694c07-1935-4a90-880e-5f1a5caf5b8f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.655759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.656419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.656703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.656977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.657638 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.659551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.689142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1210/2420] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:15:52 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:52.693309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.705674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac1fee-0690-4d34-a163-d5521b932675 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.764402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.765024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.765301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.765567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.766297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.768236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.792520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1211/2421] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:15:52 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:15:52.796763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.808177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d1766-ed23-4c4b-941d-d2ed08480110 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.813817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.814422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.814700 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.814972 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.815672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.817341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.828989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1211/2422] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:52.833104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.843815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a53576-fdb8-4df9-a3ef-e4284b6acfb2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.850088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.850739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.851014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.851298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] PATH_INFO: `/v3/projects/fe2eec4619774c6eb4b711c27faba10a/users/10d8d7e5f1de4b36a709e0fbca415ed3/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.863400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:52.866649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2eec4619774c6eb4b711c27faba10a, user_id=10d8d7e5f1de4b36a709e0fbca415ed3, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.869654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.884083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1212/2423] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:52 2022] PUT /identity/v3/projects/fe2eec4619774c6eb4b711c27faba10a/users/10d8d7e5f1de4b36a709e0fbca415ed3/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:15:52.888140 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.898070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b6db4-0464-41ba-a1cc-8b8dc5254eaf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.942007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:52.959161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d6fb-40c1-45a7-a7a1-418946db9a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.959814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d6fb-40c1-45a7-a7a1-418946db9a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.960103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d6fb-40c1-45a7-a7a1-418946db9a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.960385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d6fb-40c1-45a7-a7a1-418946db9a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.961088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d6fb-40c1-45a7-a7a1-418946db9a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.962705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d6fb-40c1-45a7-a7a1-418946db9a7e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:52.972650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1212/2424] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:15:52.976961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:52.988133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4407a-b9d9-4a9b-801a-46ed470cd380 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:52.993599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:52.994293 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:52.994571 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:52.994840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] PATH_INFO: `/v3/projects/fe2eec4619774c6eb4b711c27faba10a/users/10d8d7e5f1de4b36a709e0fbca415ed3/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:52.997028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2eec4619774c6eb4b711c27faba10a, user_id=10d8d7e5f1de4b36a709e0fbca415ed3, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:52.999909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:53.013872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1213/2425] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:15:52 2022] PUT /identity/v3/projects/fe2eec4619774c6eb4b711c27faba10a/users/10d8d7e5f1de4b36a709e0fbca415ed3/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:53.030191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:53.030449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:53.030749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:53.064405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] MFA Rules not processed for user `10d8d7e5f1de4b36a709e0fbca415ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:53.065664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:53.142370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1213/2426] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:53.154287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:53.156478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:53.215162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8d19-e6bb-4741-911b-c3b3a538420e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:53.219338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:53.219991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:53.220274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:53.220550 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:53.280235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:53.281749 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:53.292294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1214/2427] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:54.818413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:54.819975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:54.834097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9f58a-dbc4-438b-b268-6c4bd69e2982 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:54.838207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:54.838967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:54.839309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:54.839660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:54.896089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:54.897806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:54.906901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1214/2428] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:55.459803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:55.468882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce788523-1b1f-4af2-8c7a-e85944583e4a service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:55.478348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802edaa-0cbb-4809-9a07-87040f5d5f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:55.478986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802edaa-0cbb-4809-9a07-87040f5d5f80 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:55.479272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802edaa-0cbb-4809-9a07-87040f5d5f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:55.479544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802edaa-0cbb-4809-9a07-87040f5d5f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:55.489941 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802edaa-0cbb-4809-9a07-87040f5d5f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:55.491409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802edaa-0cbb-4809-9a07-87040f5d5f80 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:55.501246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1215/2429] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:15:59.396743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:59.463742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6a66-c714-4da8-a14c-4a75338a284b service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:59.468994 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:59.469895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:59.470176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:59.470413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] PATH_INFO: `/v3/users/202e4f11b5274124aaafdc140fcf5146` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:59.471629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=202e4f11b5274124aaafdc140fcf5146)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:59.473670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:59.478769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:15:59.531263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '202e4f11b5274124aaafdc140fcf5146'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:59.531984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '202e4f11b5274124aaafdc140fcf5146'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:59.539505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '202e4f11b5274124aaafdc140fcf5146'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:59.540144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '202e4f11b5274124aaafdc140fcf5146'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:59.548673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '202e4f11b5274124aaafdc140fcf5146'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:15:59.566109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1215/2430] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:15:59 2022] DELETE /identity/v3/users/202e4f11b5274124aaafdc140fcf5146 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:15:59.570759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:59.571275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:59.571368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:59.590631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:15:59.591784 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:59.652378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1216/2431] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:15:59.662691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:15:59.664852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:15:59.731566 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05924f09-71a1-40d9-95d9-c828c9113eae admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:15:59.735410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:15:59.736057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:15:59.736340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:15:59.736610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:15:59.745470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:15:59.746904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:15:59.756166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1216/2432] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:16:00.110567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:00.112388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:00.126213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fcb12-34af-4794-a412-dcce3697808a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:00.131902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:00.132612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:00.132894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:00.133171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] PATH_INFO: `/v3/projects/6c6f05fac48d4e0dbfd9b3b488b756a6` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:00.134388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6f05fac48d4e0dbfd9b3b488b756a6)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:00.136367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:00.156243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:00.190235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6f05fac48d4e0dbfd9b3b488b756a6', 'request_id': 'req-bb1e8786-2e7f-481b-9dae-455d4b3d9725'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.196028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6f05fac48d4e0dbfd9b3b488b756a6', 'request_id': 'req-bb1e8786-2e7f-481b-9dae-455d4b3d9725'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.202689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6f05fac48d4e0dbfd9b3b488b756a6', 'request_id': 'req-bb1e8786-2e7f-481b-9dae-455d4b3d9725'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.216395 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] The token cache is being invalidate because project 6c6f05fac48d4e0dbfd9b3b488b756a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:16:00.216691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.217305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-bb1e8786-2e7f-481b-9dae-455d4b3d9725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.218837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1217/2433] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:16:00 2022] DELETE /identity/v3/projects/6c6f05fac48d4e0dbfd9b3b488b756a6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:00.222967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:00.233474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ecc29-4bc7-46dc-b5b8-ba754816d067 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:00.292181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:00.292862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:00.293161 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:00.293450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] PATH_INFO: `/v3/users/60c198b9027d4ef0a46f4af2fb5acda9` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:00.294783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60c198b9027d4ef0a46f4af2fb5acda9)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:00.296767 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:00.328490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:00.350653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60c198b9027d4ef0a46f4af2fb5acda9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.351296 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60c198b9027d4ef0a46f4af2fb5acda9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.359247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60c198b9027d4ef0a46f4af2fb5acda9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.360084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60c198b9027d4ef0a46f4af2fb5acda9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.369475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60c198b9027d4ef0a46f4af2fb5acda9'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.393098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1217/2434] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:16:00 2022] DELETE /identity/v3/users/60c198b9027d4ef0a46f4af2fb5acda9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:00.432388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:00.432793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:00.433167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:00.457574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:00.459192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:00.524101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1218/2435] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:16:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:00.532998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:00.533952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:00.617643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024b811-506a-4bd7-bcee-18984895d147 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:00.623895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:00.625385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:00.625743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:00.626087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:00.636327 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:00.638006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:00.647707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1218/2436] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:16:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:16:00.734336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:00.735872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:00.818852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761aca6-c586-4e4f-ae56-b7feb97eae76 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:00.823877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:00.824608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:00.824956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:00.825290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] PATH_INFO: `/v3/projects/f8c3f3c31e1d4a839a70eef3823e5d41` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:00.827389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c3f3c31e1d4a839a70eef3823e5d41)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:00.829776 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:00.892422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c3f3c31e1d4a839a70eef3823e5d41', 'request_id': 'req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.894055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:00.906520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c3f3c31e1d4a839a70eef3823e5d41', 'request_id': 'req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.916633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c3f3c31e1d4a839a70eef3823e5d41', 'request_id': 'req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.933609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] The token cache is being invalidate because project f8c3f3c31e1d4a839a70eef3823e5d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:16:00.933609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.934401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:00.938967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1219/2437] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:16:00 2022] DELETE /identity/v3/projects/f8c3f3c31e1d4a839a70eef3823e5d41 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:00.940300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:00.956174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db95641-9d4a-4213-8204-cbec327cfd07 service swift] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:01.045453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:01.046105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:01.046386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:01.046824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] PATH_INFO: `/v3/users/10d8d7e5f1de4b36a709e0fbca415ed3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:01.048042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d8d7e5f1de4b36a709e0fbca415ed3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:01.051538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:01.105277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:01.117996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d8d7e5f1de4b36a709e0fbca415ed3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.119121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d8d7e5f1de4b36a709e0fbca415ed3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.129413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d8d7e5f1de4b36a709e0fbca415ed3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.130125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d8d7e5f1de4b36a709e0fbca415ed3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.145673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-34f82b73-09f0-4ba7-9dce-b652458665e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d8d7e5f1de4b36a709e0fbca415ed3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.171945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1219/2438] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:16:00 2022] DELETE /identity/v3/users/10d8d7e5f1de4b36a709e0fbca415ed3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:01.494491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:01.497136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:01.519793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824ce3c-e05b-4bb0-ae9d-93fc225d68b1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:01.587849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:01.588516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:01.588803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:01.589085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] PATH_INFO: `/v3/projects/fe2eec4619774c6eb4b711c27faba10a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:01.590290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2eec4619774c6eb4b711c27faba10a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:01.593667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:01.659322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2eec4619774c6eb4b711c27faba10a', 'request_id': 'req-8ad63781-7a74-4a9c-9887-bf6b67fa1187'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.667897 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2eec4619774c6eb4b711c27faba10a', 'request_id': 'req-8ad63781-7a74-4a9c-9887-bf6b67fa1187'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.675176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2eec4619774c6eb4b711c27faba10a', 'request_id': 'req-8ad63781-7a74-4a9c-9887-bf6b67fa1187'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.689572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] The token cache is being invalidate because project fe2eec4619774c6eb4b711c27faba10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:16:01.690279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.690942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-8ad63781-7a74-4a9c-9887-bf6b67fa1187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:16:01.692893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1220/2439] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:16:01 2022] DELETE /identity/v3/projects/fe2eec4619774c6eb4b711c27faba10a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:01.697961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:01.698465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:01.698602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:01.719838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:01.721167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:01.802849 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1220/2440] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:01.804987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:01.805466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:01.805792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:01.832329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:01.833609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:01.847225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1221/2441] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:01.848596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:01.858819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7662f2-e2a3-4f7d-9871-d97d0f7ac48d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:01.866963 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:01.867597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:01.867982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:01.868260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:01.868973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:16:01.869286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:01.870898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:01.884101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1221/2442] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:16:01.887923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:01.899116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c454-3691-40f1-875c-6758ca6a7508 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:01.903998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:01.904654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:01.904938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:01.905803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:01.907349 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:01.909942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:01.951409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1222/2443] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:16:01 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:16:01.952635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:01.964702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415d591-0557-491b-9cc4-abec5ceecbaf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:02.045881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:02.046513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:02.046793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:02.047068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:02.047738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:02.049822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:02.074677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1222/2444] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:16:01 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:16:02.078594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:02.089971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdd1c4-bf3e-4083-a570-14678451d7e4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:02.097046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:02.097831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:02.098285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:02.098608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:02.099387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:02.101178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:02.115369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1223/2445] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:16:02.117119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:02.128508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d40e9-97c3-4260-8a9b-96b8f4b9475e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:02.135882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9394b-8040-4d91-8af7-1eda34e35a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:02.135882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9394b-8040-4d91-8af7-1eda34e35a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:02.135882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9394b-8040-4d91-8af7-1eda34e35a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:02.135882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9394b-8040-4d91-8af7-1eda34e35a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:02.136405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9394b-8040-4d91-8af7-1eda34e35a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:02.137013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9394b-8040-4d91-8af7-1eda34e35a2e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:02.148366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1223/2446] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:16:02.152593 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:02.165950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df96bb6-dee6-4c06-a3e0-04c78fbdabf1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:02.173623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:02.174347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:02.174664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:02.174948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] PATH_INFO: `/v3/projects/28b64940359146368434f38f443b5afd/users/65d6d5c6915543dfac67beef1ddaf8b3/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:02.191524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b64940359146368434f38f443b5afd, user_id=65d6d5c6915543dfac67beef1ddaf8b3, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:02.194609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:02.210459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1224/2447] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:16:02 2022] PUT /identity/v3/projects/28b64940359146368434f38f443b5afd/users/65d6d5c6915543dfac67beef1ddaf8b3/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:02.215862 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025f4f-b085-4752-b716-f6626f5d2dd1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:02.216373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025f4f-b085-4752-b716-f6626f5d2dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:02.216803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025f4f-b085-4752-b716-f6626f5d2dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:02.239971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-cc025f4f-b085-4752-b716-f6626f5d2dd1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:02.241558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-cc025f4f-b085-4752-b716-f6626f5d2dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:02.257128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:02.324845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1224/2448] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:02.335269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:02.337066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:02.350715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:02.414598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c1a4b-352e-408c-92a6-e56488d6d7f7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:02.418857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:02.419006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:02.419164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:02.419440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:02.436033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:02.437342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:02.461125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1225/2449] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:16:07.549405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:07.549934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:07.550019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:07.582716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] MFA Rules not processed for user `65d6d5c6915543dfac67beef1ddaf8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:07.584806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:07.719865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1225/2450] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:16:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:07.721274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:07.733979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af96cf-9fd5-45a2-b5ef-75fbbf2fcade service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:07.743840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:07.745411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:07.745883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:07.746338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:07.747200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:07.749426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:07.802636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1226/2451] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:16:07 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:16:07.804021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:07.823312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505be0a-9a43-4d95-a55d-6fa60595a9ef None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:07.917366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:07.918238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:07.918712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:07.919182 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:07.920096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:07.922438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:07.953932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1226/2452] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:16:07 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:16:07.957509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:07.980690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e997f4-9086-48d9-b6f2-346152928ef2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:07.989425 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:07.990344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:07.990696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:07.991102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:07.992062 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:07.996503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:08.020729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1227/2453] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:16:08.025251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:08.041617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817429c-fd13-46e1-8f0c-1aa201072edc admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:08.049781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:08.050496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:08.050768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:08.051045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] PATH_INFO: `/v3/projects/2b3e1ed1c79d41e09a2919104e30ad6f/users/a286bdb63349463c897917c3abcbc5c3/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:08.080010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3e1ed1c79d41e09a2919104e30ad6f, user_id=a286bdb63349463c897917c3abcbc5c3, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:08.082948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:08.103069 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1227/2454] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:16:08 2022] PUT /identity/v3/projects/2b3e1ed1c79d41e09a2919104e30ad6f/users/a286bdb63349463c897917c3abcbc5c3/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:16:08.106994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:08.119655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01444456-f395-4224-8a58-2c80216d7b64 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:08.217936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b089-9527-4843-aa10-81f3c1d1f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:08.218901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b089-9527-4843-aa10-81f3c1d1f66b admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:08.219243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b089-9527-4843-aa10-81f3c1d1f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:08.219579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b089-9527-4843-aa10-81f3c1d1f66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:08.220565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b089-9527-4843-aa10-81f3c1d1f66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:08.222276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b089-9527-4843-aa10-81f3c1d1f66b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:08.234352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1228/2455] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:16:08.238513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:08.251031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9abd-f5d8-4b15-810e-6d2d8ddeec34 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:08.255938 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:08.256674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:08.256953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:08.257228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] PATH_INFO: `/v3/projects/2b3e1ed1c79d41e09a2919104e30ad6f/users/a286bdb63349463c897917c3abcbc5c3/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:08.259255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3e1ed1c79d41e09a2919104e30ad6f, user_id=a286bdb63349463c897917c3abcbc5c3, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:08.266521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:08.276027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1228/2456] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:16:08 2022] PUT /identity/v3/projects/2b3e1ed1c79d41e09a2919104e30ad6f/users/a286bdb63349463c897917c3abcbc5c3/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:16:14.105182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ca00b-f442-4daf-8ca2-9e95fd463519 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:14.105844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ca00b-f442-4daf-8ca2-9e95fd463519 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:14.105844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ca00b-f442-4daf-8ca2-9e95fd463519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:14.135903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b74ca00b-f442-4daf-8ca2-9e95fd463519 None None] MFA Rules not processed for user `a286bdb63349463c897917c3abcbc5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:14.137453 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b74ca00b-f442-4daf-8ca2-9e95fd463519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:14.244126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:14.266058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1229/2457] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:16:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:14.278294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:14.281039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:14.286785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:14.358340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbdcd8-cb6c-4954-9e14-78d5490804b8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:14.364375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:14.365135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:14.365478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:14.365818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:14.470037 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:14.472369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:14.484486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1229/2458] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:16:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:16:14.646160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eb55d-6207-4e40-bd00-e2c4392c4ed9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:14.646589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eb55d-6207-4e40-bd00-e2c4392c4ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:14.646969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eb55d-6207-4e40-bd00-e2c4392c4ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:14.822787 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b64eb55d-6207-4e40-bd00-e2c4392c4ed9 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:14.825935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b64eb55d-6207-4e40-bd00-e2c4392c4ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:14.895189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1230/2459] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:16:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:14.912070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:14.914029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:15.014921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2a03-2069-4637-a362-0474d3af9b93 service glance] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:15.021257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182a701-7be9-4f71-a036-c7b0b956e56f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:15.022071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182a701-7be9-4f71-a036-c7b0b956e56f service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:15.022499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182a701-7be9-4f71-a036-c7b0b956e56f service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:15.022872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182a701-7be9-4f71-a036-c7b0b956e56f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:15.032629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182a701-7be9-4f71-a036-c7b0b956e56f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:15.034385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182a701-7be9-4f71-a036-c7b0b956e56f service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:15.043817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1230/2460] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:16:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:16:19.026411 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684eace-0205-4fc4-8d2f-ff65f4404627 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.027036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684eace-0205-4fc4-8d2f-ff65f4404627 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.027036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684eace-0205-4fc4-8d2f-ff65f4404627 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.028364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1231/2461] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:16:19.036800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622ad3-5d69-4733-b8db-a764bd3af13f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.037395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622ad3-5d69-4733-b8db-a764bd3af13f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.037395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622ad3-5d69-4733-b8db-a764bd3af13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.058189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eb622ad3-5d69-4733-b8db-a764bd3af13f None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:19.061619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eb622ad3-5d69-4733-b8db-a764bd3af13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:19.136583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:19.191802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1231/2462] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:16:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:19.212203 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684eace-0205-4fc4-8d2f-ff65f4404627 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:19.224027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684eace-0205-4fc4-8d2f-ff65f4404627 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:19.227688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e989ad-f0e3-4bc6-bcbf-4b96935c3c90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:19.230642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e989ad-f0e3-4bc6-bcbf-4b96935c3c90 service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.230925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e989ad-f0e3-4bc6-bcbf-4b96935c3c90 service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.231200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e989ad-f0e3-4bc6-bcbf-4b96935c3c90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.242830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e989ad-f0e3-4bc6-bcbf-4b96935c3c90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:19.244351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e989ad-f0e3-4bc6-bcbf-4b96935c3c90 service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:19.264375 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1232/2463] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:16:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:16:19.347324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722b4c7-0d49-4bfe-8609-708f4cd6f473 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.347597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722b4c7-0d49-4bfe-8609-708f4cd6f473 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.347912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722b4c7-0d49-4bfe-8609-708f4cd6f473 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.349282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1232/2464] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:16:19.354141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b30f96-5df1-4ce6-a94b-2bbfb7556493 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.354424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b30f96-5df1-4ce6-a94b-2bbfb7556493 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.354742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b30f96-5df1-4ce6-a94b-2bbfb7556493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.376387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f8b30f96-5df1-4ce6-a94b-2bbfb7556493 None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:19.377664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f8b30f96-5df1-4ce6-a94b-2bbfb7556493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:19.435176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1233/2465] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:16:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:19.448682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722b4c7-0d49-4bfe-8609-708f4cd6f473 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:19.461978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722b4c7-0d49-4bfe-8609-708f4cd6f473 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:19.465686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f474a-d97f-490f-9b81-da02ab93a33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:19.466671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f474a-d97f-490f-9b81-da02ab93a33e service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.466671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f474a-d97f-490f-9b81-da02ab93a33e service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.467338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f474a-d97f-490f-9b81-da02ab93a33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.475293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f474a-d97f-490f-9b81-da02ab93a33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:19.476848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f474a-d97f-490f-9b81-da02ab93a33e service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:19.487851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1233/2466] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:16:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:16:19.586287 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f5e1-d3a3-4a06-8a98-05cc1fd50feb None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.586582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f5e1-d3a3-4a06-8a98-05cc1fd50feb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.586854 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f5e1-d3a3-4a06-8a98-05cc1fd50feb None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.588286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1234/2467] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:16:19.594364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.594650 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.594919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.619094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:19.620358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:19.632939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1234/2468] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:16:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:19.641708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750f5e1-d3a3-4a06-8a98-05cc1fd50feb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:19.643541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4750f5e1-d3a3-4a06-8a98-05cc1fd50feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:19.658944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750f5e1-d3a3-4a06-8a98-05cc1fd50feb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:19.662972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07a487-9c3e-4a24-a04b-4f6add3129c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:19.663970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07a487-9c3e-4a24-a04b-4f6add3129c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:19.664257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07a487-9c3e-4a24-a04b-4f6add3129c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:19.664528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07a487-9c3e-4a24-a04b-4f6add3129c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:19.675693 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07a487-9c3e-4a24-a04b-4f6add3129c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:19.677261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07a487-9c3e-4a24-a04b-4f6add3129c1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:19.692319 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1235/2469] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:16:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:16:21.168499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:21.173714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:21.262072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5045d9-8f0d-4bb5-aa0c-a6f769d67d6e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:21.267317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:21.268137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:21.268137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:21.268410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:21.270306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:21.282676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:21.284321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:21.295384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1235/2470] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:16:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:16:23.034178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327d4b3-3156-4363-a73f-8881f77696dc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:23.034677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327d4b3-3156-4363-a73f-8881f77696dc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:23.034677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327d4b3-3156-4363-a73f-8881f77696dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:23.058361 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6327d4b3-3156-4363-a73f-8881f77696dc None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:23.059588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6327d4b3-3156-4363-a73f-8881f77696dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:23.069826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1236/2471] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:16:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:23.091754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:23.108072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c066e94-8cbd-4433-8b4e-80cc39125e36 service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:23.114195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de173f30-eaaf-47f7-9633-bf75663e759d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:23.114844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173f30-eaaf-47f7-9633-bf75663e759d service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:23.115109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173f30-eaaf-47f7-9633-bf75663e759d service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:23.115391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173f30-eaaf-47f7-9633-bf75663e759d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:23.125175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de173f30-eaaf-47f7-9633-bf75663e759d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:23.126492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de173f30-eaaf-47f7-9633-bf75663e759d service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:23.135681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1236/2472] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:16:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:16:26.024329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742766-17df-4a67-a823-060433b773a9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:26.025180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742766-17df-4a67-a823-060433b773a9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:26.029793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742766-17df-4a67-a823-060433b773a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:26.049517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8e742766-17df-4a67-a823-060433b773a9 None None] MFA Rules not processed for user `65d6d5c6915543dfac67beef1ddaf8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:26.050760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e742766-17df-4a67-a823-060433b773a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:26.058903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1237/2473] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:16:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:33.131966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=57 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:33.945980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a2900-8bc8-49f6-b17e-78fc1ed30851 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:33.946499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a2900-8bc8-49f6-b17e-78fc1ed30851 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:33.946784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a2900-8bc8-49f6-b17e-78fc1ed30851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:33.966666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-511a2900-8bc8-49f6-b17e-78fc1ed30851 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:33.968012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-511a2900-8bc8-49f6-b17e-78fc1ed30851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:33.975908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1237/2474] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:16:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:34.002584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742766-17df-4a67-a823-060433b773a9 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:34.020801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742766-17df-4a67-a823-060433b773a9 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:34.026093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:34.027192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:34.027420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:34.027830 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:34.037031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:34.039149 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:34.048184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1238/2475] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:16:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:16:43.961950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:44.044255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=74 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:16:51.151091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:51.151576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:51.151665 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:51.172267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:16:51.173465 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:16:51.181128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1238/2476] 10.222.0.30 () {58 vars in 967 bytes} [Fri Aug 5 05:16:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:16:51.194004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:16:51.204907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31789e-b4eb-48f3-9899-7067c580610e service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:16:51.210173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccd4cc-2baa-4fab-b85a-2672c1be3106 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:16:51.210783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd4cc-2baa-4fab-b85a-2672c1be3106 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:16:51.211065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd4cc-2baa-4fab-b85a-2672c1be3106 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:16:51.211308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd4cc-2baa-4fab-b85a-2672c1be3106 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:16:51.219435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccd4cc-2baa-4fab-b85a-2672c1be3106 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:16:51.221041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccd4cc-2baa-4fab-b85a-2672c1be3106 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:16:51.229000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1239/2477] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:16:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:17:01.166342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:17:01.225986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:17:04.356567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:17:04.366993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfc53e-037a-42fd-a290-47da72e57514 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:17:04.371200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd671b-8a6b-4325-9fa3-480fb8087ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:17:04.371895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd671b-8a6b-4325-9fa3-480fb8087ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:17:04.372177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd671b-8a6b-4325-9fa3-480fb8087ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:17:04.372457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd671b-8a6b-4325-9fa3-480fb8087ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:17:04.383354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd671b-8a6b-4325-9fa3-480fb8087ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:17:04.384836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd671b-8a6b-4325-9fa3-480fb8087ab0 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:17:04.394912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1239/2478] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:17:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:17:12.152634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65553e8b-fe07-4922-9b41-e43fb02ca684 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:17:12.153221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65553e8b-fe07-4922-9b41-e43fb02ca684 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:17:12.153221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65553e8b-fe07-4922-9b41-e43fb02ca684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:17:12.173485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-65553e8b-fe07-4922-9b41-e43fb02ca684 None None] MFA Rules not processed for user `c7c57f5e11a84bf98d474837adbb0ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:17:12.174718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-65553e8b-fe07-4922-9b41-e43fb02ca684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:17:12.241709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1240/2479] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:17:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:17:14.390438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:17:22.234456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=25 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:17:43.210460 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:17:43.210991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:17:43.210991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:17:43.230309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] MFA Rules not processed for user `b89279618557486f9d5b358b654a906e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:17:43.231504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:17:43.288310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1240/2480] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:17:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:17:49.636339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:17:49.636861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:17:49.636861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:17:49.656200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] MFA Rules not processed for user `04eb412804c247908f8d377a42d1ea63`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:17:49.657409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:17:49.721956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1241/2481] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:17:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:17:53.280409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=25 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:17:59.715239 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=25 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:01.183992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:01.194470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf63bd-2c74-4909-9157-896e05a66ef5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:01.199065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:01.199675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:01.199983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:01.200257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:01.202376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:01.216178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:01.217609 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:01.228692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1241/2482] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:03.544506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:03.554609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796da1b0-aacf-430c-b162-c5aba828849a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:03.559302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:03.560301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:03.560586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:03.560849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:03.562655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:03.579115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:03.580596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:03.591965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1242/2483] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:11.223707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:13.586967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:15.744089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:15.753461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf001b-8388-4ae8-bff3-55e196947482 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:15.758133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:15.758930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:15.759072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:15.759258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:15.761437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:15.831203 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:15.832684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:15.842306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1242/2484] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:19.007748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.009323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:19.024686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a788e6-e574-4923-9cad-bc469b33db7d service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.029560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.030209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.030493 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.030774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] PATH_INFO: `/v3/users/c7c57f5e11a84bf98d474837adbb0ba7` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.032197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c57f5e11a84bf98d474837adbb0ba7)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.034156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.084341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c57f5e11a84bf98d474837adbb0ba7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.085046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c57f5e11a84bf98d474837adbb0ba7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.092090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c57f5e11a84bf98d474837adbb0ba7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.092777 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c57f5e11a84bf98d474837adbb0ba7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.102560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b861e5e6-b645-4c35-93d9-41300232c89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c57f5e11a84bf98d474837adbb0ba7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.120071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1243/2485] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:18:19 2022] DELETE /identity/v3/users/c7c57f5e11a84bf98d474837adbb0ba7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:19.296284 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.308117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253c03-7d54-4200-ad58-5dfc759d24b9 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.370523 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.371155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.371434 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.371704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] PATH_INFO: `/v3/projects/4b672d5801cb4fdf87d6626f4603e281` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.372918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b672d5801cb4fdf87d6626f4603e281)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.374817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.423004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b672d5801cb4fdf87d6626f4603e281', 'request_id': 'req-08b5d228-ffeb-46b6-951f-4470ef5590f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.428775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b672d5801cb4fdf87d6626f4603e281', 'request_id': 'req-08b5d228-ffeb-46b6-951f-4470ef5590f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.434866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b672d5801cb4fdf87d6626f4603e281', 'request_id': 'req-08b5d228-ffeb-46b6-951f-4470ef5590f8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.448302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] The token cache is being invalidate because project 4b672d5801cb4fdf87d6626f4603e281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:18:19.448598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.449250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-08b5d228-ffeb-46b6-951f-4470ef5590f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:19.450763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1243/2486] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:18:19 2022] DELETE /identity/v3/projects/4b672d5801cb4fdf87d6626f4603e281 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:19.456117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.456406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.456674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.477789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:19.479009 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:19.539867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1244/2487] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:18:19.548277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.548412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.548610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.570131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:19.571363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:19.580743 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1244/2488] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:18:19.585196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.594839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e1434-bfb0-4b8a-bf0a-f99944b0a1a2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.599662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.600372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.600657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.600926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.601616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:18:19.601918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.603509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.614831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1245/2489] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:18:19.619131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.629054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe65b8-794a-43c4-82b6-e56aacac693d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.634128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.634748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.635036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.635309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.636038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.637935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.654323 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:19.669933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1245/2490] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:18:19 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:18:19.674122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.684018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac2865-e2bd-48b7-abe1-fc8853366150 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.740200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.740823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.741099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.741365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.742015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.743956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.754989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:19.771081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1246/2491] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:18:19 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:18:19.775324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.786279 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9abb9-4144-49b3-9fc5-8318b9e7afba admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.792019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.792634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.792908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.793181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.793875 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.795474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.805866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1246/2492] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:18:19.809867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.820466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41e14-19b7-456a-9c57-4099ff8a9da7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.826071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488657ec-263c-403b-acc5-805e5a997f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.826668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488657ec-263c-403b-acc5-805e5a997f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.826946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488657ec-263c-403b-acc5-805e5a997f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.827205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488657ec-263c-403b-acc5-805e5a997f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.827920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488657ec-263c-403b-acc5-805e5a997f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.829518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488657ec-263c-403b-acc5-805e5a997f85 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.840567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1247/2493] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:18:19.844755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:19.855962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd815f4c-723c-4de7-828f-4a674b215c78 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:19.860924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:19.861606 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.861889 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.862167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] PATH_INFO: `/v3/projects/e45c617eb5804c8187985b40e94329d7/users/2637eface78b4813b91967ce66a59c00/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.877981 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45c617eb5804c8187985b40e94329d7, user_id=2637eface78b4813b91967ce66a59c00, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:19.880777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:19.895600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1247/2494] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:18:19 2022] PUT /identity/v3/projects/e45c617eb5804c8187985b40e94329d7/users/2637eface78b4813b91967ce66a59c00/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:19.900858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02c84-32e4-406e-aa6a-09eac98120bc None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:19.901149 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02c84-32e4-406e-aa6a-09eac98120bc None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:19.901429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02c84-32e4-406e-aa6a-09eac98120bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:19.921167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-abd02c84-32e4-406e-aa6a-09eac98120bc None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:19.922387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-abd02c84-32e4-406e-aa6a-09eac98120bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:19.991460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1248/2495] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:18:20.001493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:20.003329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:20.067541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d801236-686e-4929-b891-e3beea936240 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:20.073383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:20.074247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:20.074555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:20.074932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:20.087685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:20.089136 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:20.098308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1248/2496] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:24.044133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea277d2-f4ba-4974-9d7b-c8628f8622c9 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:24.044582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea277d2-f4ba-4974-9d7b-c8628f8622c9 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:24.044804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea277d2-f4ba-4974-9d7b-c8628f8622c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:24.074442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1ea277d2-f4ba-4974-9d7b-c8628f8622c9 None None] MFA Rules not processed for user `2637eface78b4813b91967ce66a59c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:24.075843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1ea277d2-f4ba-4974-9d7b-c8628f8622c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:24.148620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1249/2497] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:18:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:18:24.163259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:24.164705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:24.275597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272c5b3-67fd-4508-bcf9-0d5f996ce45c service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:24.282681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:24.283421 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:24.283782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:24.284145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:24.295708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:24.297382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:24.309350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1249/2498] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:25.561796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74516de2-b824-4f59-9545-966f8d52786f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:25.562328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74516de2-b824-4f59-9545-966f8d52786f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:25.562455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74516de2-b824-4f59-9545-966f8d52786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:25.581694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-74516de2-b824-4f59-9545-966f8d52786f None None] MFA Rules not processed for user `2637eface78b4813b91967ce66a59c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:25.582921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-74516de2-b824-4f59-9545-966f8d52786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:25.591570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1250/2499] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:18:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:18:25.916726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:25.918398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:25.931995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455eea-2562-4ac4-95de-75bb08892c52 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:25.936169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:25.936813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:25.937076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:25.937348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:25.939234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:25.950121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:25.951565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:25.960217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1250/2500] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:26.303560 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74516de2-b824-4f59-9545-966f8d52786f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:26.305357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-74516de2-b824-4f59-9545-966f8d52786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:26.397087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:26.402924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74516de2-b824-4f59-9545-966f8d52786f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:26.416038 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:26.416767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:26.417139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:26.417478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:26.419382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:26.431655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:26.433356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:26.444128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1251/2501] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:32.091958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:32.101817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343cf4f-3daf-4c2e-90bb-3fa7c1d02631 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:32.105601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:32.106248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:32.106516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:32.106785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:32.108888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:32.119744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:32.121207 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da1aa-6c84-44d8-b484-c03e0d48501c service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:32.131107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1251/2502] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:35.769074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:35.782070 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60f99-a573-4b38-a7da-c42520b46b70 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:35.789301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:35.789930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:35.790659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:35.790659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:35.792256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:35.856186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:35.858132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:35.867792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1252/2503] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:40.421709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:40.422173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:40.422278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:40.442246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:40.443481 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:40.461728 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:40.504962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1252/2504] 10.222.0.30 () {58 vars in 967 bytes} [Fri Aug 5 05:18:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:18:40.553594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:40.555073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:40.622948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dba4b3-68c7-4a2a-81df-7dc4a03622ad service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:40.627855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a577f8-1c05-476b-87f1-246300a5d999 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:40.628476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a577f8-1c05-476b-87f1-246300a5d999 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:40.628725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a577f8-1c05-476b-87f1-246300a5d999 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:40.628993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a577f8-1c05-476b-87f1-246300a5d999 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:40.638007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a577f8-1c05-476b-87f1-246300a5d999 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:40.639645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a577f8-1c05-476b-87f1-246300a5d999 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:40.647455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1253/2505] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:18:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:18:40.848031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:40.849800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:40.867071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c546097-0f90-4cbe-b8fd-cbf087b0feb6 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:40.876825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:40.877756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:40.878152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:40.878412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:40.880556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:40.893809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:40.895446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906e149-6b29-419e-9a45-cb3515170315 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:40.908901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1253/2506] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:18:44.190856 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46bfa3-5e30-4792-8fc1-159e4bf97883 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:44.191419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46bfa3-5e30-4792-8fc1-159e4bf97883 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:44.191529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46bfa3-5e30-4792-8fc1-159e4bf97883 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:44.193084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1254/2507] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:18:44.202724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:44.203267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:44.203267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:44.223453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] MFA Rules not processed for user `7b3c6d4d4371427ca4ca74a962a47967`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:18:44.224681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:44.234799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1254/2508] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:18:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:18:44.260526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46bfa3-5e30-4792-8fc1-159e4bf97883 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:44.263015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bf46bfa3-5e30-4792-8fc1-159e4bf97883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:44.305377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46bfa3-5e30-4792-8fc1-159e4bf97883 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:44.310966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:44.311732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:44.312115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:44.312456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:44.321596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:44.323276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:44.333762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1255/2509] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:46.685256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:46.699792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbdba7-d496-4cc7-8a7b-0cdc03c301c9 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:46.707048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:46.708267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:46.708775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:46.709310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:46.711404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:46.778561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:46.781109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:46.791501 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1255/2510] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:18:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:18:47.481042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:47.481786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:47.499335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22aaa69-d293-4fad-945c-17c7162e130e service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:47.504019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:47.504667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:47.504998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:47.505216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] PATH_INFO: `/v3/users/65d6d5c6915543dfac67beef1ddaf8b3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:47.506401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d6d5c6915543dfac67beef1ddaf8b3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:47.508638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:47.563143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d6d5c6915543dfac67beef1ddaf8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.563843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d6d5c6915543dfac67beef1ddaf8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.571569 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d6d5c6915543dfac67beef1ddaf8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.572471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d6d5c6915543dfac67beef1ddaf8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.576577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:47.582997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d6d5c6915543dfac67beef1ddaf8b3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.602565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1256/2511] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:18:47 2022] DELETE /identity/v3/users/65d6d5c6915543dfac67beef1ddaf8b3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:47.788845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:47.799897 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da6d0-9fcb-4b4b-9ca3-c9830fdc37fd service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:47.855567 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:47.856346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:47.856577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:47.856858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] PATH_INFO: `/v3/projects/28b64940359146368434f38f443b5afd` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:47.858415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b64940359146368434f38f443b5afd)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:47.860404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:47.906895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:47.908480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b64940359146368434f38f443b5afd', 'request_id': 'req-85ec89ae-6148-403d-9392-bff70f641eb1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.915109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b64940359146368434f38f443b5afd', 'request_id': 'req-85ec89ae-6148-403d-9392-bff70f641eb1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.921338 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b64940359146368434f38f443b5afd', 'request_id': 'req-85ec89ae-6148-403d-9392-bff70f641eb1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.935703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] The token cache is being invalidate because project 28b64940359146368434f38f443b5afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:18:47.936056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.936834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:47.938717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1256/2512] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:18:47 2022] DELETE /identity/v3/projects/28b64940359146368434f38f443b5afd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:47.942868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:47.944397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:47.957341 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51409802-08db-4e3c-8b4b-19b2784a447a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:48.018086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:48.018736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:48.019013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:48.019281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] PATH_INFO: `/v3/users/a286bdb63349463c897917c3abcbc5c3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:48.020476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a286bdb63349463c897917c3abcbc5c3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:48.022409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:48.076837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a286bdb63349463c897917c3abcbc5c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.077524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a286bdb63349463c897917c3abcbc5c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.084666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a286bdb63349463c897917c3abcbc5c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.085305 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a286bdb63349463c897917c3abcbc5c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.093500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cdf5206a-cfa3-40ea-8f3f-72feb16343f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a286bdb63349463c897917c3abcbc5c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.110182 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1257/2513] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:18:47 2022] DELETE /identity/v3/users/a286bdb63349463c897917c3abcbc5c3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:48.313952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:18:48.315283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:18:48.330177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec89ae-6148-403d-9392-bff70f641eb1 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:18:48.395426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:18:48.396188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:18:48.397011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:18:48.397315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] PATH_INFO: `/v3/projects/2b3e1ed1c79d41e09a2919104e30ad6f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:18:48.399010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3e1ed1c79d41e09a2919104e30ad6f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:18:48.401036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:18:48.452113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3e1ed1c79d41e09a2919104e30ad6f', 'request_id': 'req-e93ab137-6b39-42ec-9e25-180959ae50a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.457646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3e1ed1c79d41e09a2919104e30ad6f', 'request_id': 'req-e93ab137-6b39-42ec-9e25-180959ae50a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.465196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3e1ed1c79d41e09a2919104e30ad6f', 'request_id': 'req-e93ab137-6b39-42ec-9e25-180959ae50a8'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.478805 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] The token cache is being invalidate because project 2b3e1ed1c79d41e09a2919104e30ad6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:18:48.479236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.480070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:18:48.482012 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1257/2514] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:18:48 2022] DELETE /identity/v3/projects/2b3e1ed1c79d41e09a2919104e30ad6f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:18:58.106760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:18:58.475489 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=58,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:03.017030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:03.017684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:03.018031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:03.039012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:03.040371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:03.099929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1258/2515] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:19:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:03.111693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:03.113147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:03.184352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab137-6b39-42ec-9e25-180959ae50a8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:03.190584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:03.191216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:03.191486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:03.191789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:03.200453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:03.202085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:03.210838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1258/2516] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:19:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:19:03.500601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:03.502096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:03.601390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035aa5e-cbba-44f3-95c2-17764a7c13a4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:03.607086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:03.608069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:03.608410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:03.608752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:03.610518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:03.623886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:03.625498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611402ca-4dcc-4a91-adb2-565ff822d960 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:03.637702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1259/2517] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:13.207381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=32 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:13.416142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:13.417571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:13.433112 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23c7d-0dd1-4ab5-8267-e31d383e3530 service swift] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:13.438930 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:13.439707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:13.440039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:13.440324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:13.442318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:13.455266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:13.456954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139db7b-4de6-4d9c-9b37-6fb6108f64bd service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:13.468693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1259/2518] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:13.632146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=59 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:16.342156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905971-879f-40a5-8e33-33752c3c2ba8 None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.342836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905971-879f-40a5-8e33-33752c3c2ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.342836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905971-879f-40a5-8e33-33752c3c2ba8 None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.344406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1260/2519] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:19:16.350943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518b49-a01b-42bc-9c4b-92bc966e4109 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.351475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518b49-a01b-42bc-9c4b-92bc966e4109 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.351475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518b49-a01b-42bc-9c4b-92bc966e4109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.372660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b4518b49-a01b-42bc-9c4b-92bc966e4109 None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:16.373903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b4518b49-a01b-42bc-9c4b-92bc966e4109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:16.383350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1260/2520] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:19:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:16.395936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45905971-879f-40a5-8e33-33752c3c2ba8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:16.397611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-45905971-879f-40a5-8e33-33752c3c2ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:16.475145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45905971-879f-40a5-8e33-33752c3c2ba8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:16.480152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2f9a0-3cb5-46a5-9200-552dac4782cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:16.481178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f9a0-3cb5-46a5-9200-552dac4782cf service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.481674 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f9a0-3cb5-46a5-9200-552dac4782cf service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.482043 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f9a0-3cb5-46a5-9200-552dac4782cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.494652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f9a0-3cb5-46a5-9200-552dac4782cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:16.496228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f9a0-3cb5-46a5-9200-552dac4782cf service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:16.509122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1261/2521] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:16.580870 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f007ca5-56c4-4590-a122-b0432509f0f0 None None] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.581241 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f007ca5-56c4-4590-a122-b0432509f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.581577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f007ca5-56c4-4590-a122-b0432509f0f0 None None] PATH_INFO: `/` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.583229 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1261/2522] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:19:16.588274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce843f-80e2-40bb-8f08-040b86f64622 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.588565 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce843f-80e2-40bb-8f08-040b86f64622 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.588840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce843f-80e2-40bb-8f08-040b86f64622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.609398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0fce843f-80e2-40bb-8f08-040b86f64622 None None] MFA Rules not processed for user `cfb671f152cd4c40825a81eec15af784`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:16.610628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0fce843f-80e2-40bb-8f08-040b86f64622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:16.665466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1262/2523] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:19:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:16.680135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f007ca5-56c4-4590-a122-b0432509f0f0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:16.682380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2f007ca5-56c4-4590-a122-b0432509f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:16.695706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f007ca5-56c4-4590-a122-b0432509f0f0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:16.699317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb81c5-ed7e-4b63-b2a9-e3a47cb5d320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:16.699993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb81c5-ed7e-4b63-b2a9-e3a47cb5d320 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.700270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb81c5-ed7e-4b63-b2a9-e3a47cb5d320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.700787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb81c5-ed7e-4b63-b2a9-e3a47cb5d320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.707906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb81c5-ed7e-4b63-b2a9-e3a47cb5d320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:16.709364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb81c5-ed7e-4b63-b2a9-e3a47cb5d320 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:16.717049 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1262/2524] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:16.822652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e94b7-190b-4c4c-9476-1c6eef71f3ab None None] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.823021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e94b7-190b-4c4c-9476-1c6eef71f3ab None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.823365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e94b7-190b-4c4c-9476-1c6eef71f3ab None None] PATH_INFO: `/` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.825118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1263/2525] 10.222.0.30 () {54 vars in 921 bytes} [Fri Aug 5 05:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 05:19:16.830200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4863b-94c9-49ab-95a5-bde2997f4950 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.830494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4863b-94c9-49ab-95a5-bde2997f4950 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.830770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4863b-94c9-49ab-95a5-bde2997f4950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.851246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-16c4863b-94c9-49ab-95a5-bde2997f4950 None None] MFA Rules not processed for user `5134c03bb8854cd4bee58644e6eacc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:16.852471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-16c4863b-94c9-49ab-95a5-bde2997f4950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:16.862114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1263/2526] 10.222.0.30 () {58 vars in 1018 bytes} [Fri Aug 5 05:19:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:16.875521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e94b7-190b-4c4c-9476-1c6eef71f3ab None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:16.877600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7b3e94b7-190b-4c4c-9476-1c6eef71f3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:16.893892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e94b7-190b-4c4c-9476-1c6eef71f3ab None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:16.898100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:16.898765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:16.899114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:16.899390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:16.910762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:16.912609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:16.924268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1264/2527] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:18.865418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:18.865965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:18.865965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:18.886415 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:18.888330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:18.896449 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1264/2528] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:19:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:18.912161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:18.923383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6f6d6-0738-472a-b700-9a33ee533abc service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:18.928817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:18.929415 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:18.929692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:18.929964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:18.938106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:18.939792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:18.947937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1265/2529] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:19:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:19:20.628701 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:20.638077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dd4b6-3145-4a2b-be5d-6693a182eb97 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:20.641976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:20.642654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:20.642989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:20.643282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:20.645199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:20.711637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:20.713783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:20.724585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1265/2530] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:24.924271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:24.926482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:24.941128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef984d-87e4-4af3-ada7-68727643572b service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:24.946284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:24.946939 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:24.947225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:24.947504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] PATH_INFO: `/v3/users/04eb412804c247908f8d377a42d1ea63` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:24.949664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04eb412804c247908f8d377a42d1ea63)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:24.951618 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:25.014904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04eb412804c247908f8d377a42d1ea63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.015673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04eb412804c247908f8d377a42d1ea63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.023213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04eb412804c247908f8d377a42d1ea63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.023931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04eb412804c247908f8d377a42d1ea63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.033978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3ee6a6a9-c9bb-4f7c-8e89-ff62f11917b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04eb412804c247908f8d377a42d1ea63'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.058617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:25.071781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1266/2531] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:24 2022] DELETE /identity/v3/users/04eb412804c247908f8d377a42d1ea63 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:25.360468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:25.378020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf4374-b79d-43e3-9e7a-1511d4172be4 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:25.460259 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:25.461222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.461505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.464122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] PATH_INFO: `/v3/projects/27c58f39e03f42d5862a6a64194f5711` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.465348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c58f39e03f42d5862a6a64194f5711)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:25.467511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:25.536749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c58f39e03f42d5862a6a64194f5711', 'request_id': 'req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.537966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:25.543914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c58f39e03f42d5862a6a64194f5711', 'request_id': 'req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.553282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c58f39e03f42d5862a6a64194f5711', 'request_id': 'req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.568436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] The token cache is being invalidate because project 27c58f39e03f42d5862a6a64194f5711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:25.568748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.569513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0eaa78a4-0a0d-4f0f-8d41-305e53c7dd56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:25.571226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1266/2532] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:25 2022] DELETE /identity/v3/projects/27c58f39e03f42d5862a6a64194f5711 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:25.576366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.577610 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.578065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.598781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:25.600141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:25.672829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1267/2533] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:25.681467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746344-04ab-4678-af38-185975f3b4ce None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.681755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746344-04ab-4678-af38-185975f3b4ce None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.682030 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746344-04ab-4678-af38-185975f3b4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.705533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-40746344-04ab-4678-af38-185975f3b4ce None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:25.708257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40746344-04ab-4678-af38-185975f3b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:25.717861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1267/2534] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:25.724514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:25.740025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08ccd4-9ce8-4659-916e-56321fcd9778 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:25.747967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:25.751507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.751898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.752080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.752822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:19:25.753140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:25.754774 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:25.770799 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1268/2535] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:19:25.775395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746344-04ab-4678-af38-185975f3b4ce None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:25.784901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746344-04ab-4678-af38-185975f3b4ce None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:25.790311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:25.790962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.791275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.791553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.792340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:25.794355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:25.830614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1268/2536] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:25 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:25.831812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:25.841892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a58bca-36ea-4103-8b32-6dabcf84f7f4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:25.905368 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:25.906265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.906708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.907131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.908192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:25.910316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:25.932940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1269/2537] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:25 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:19:25.937077 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:25.948766 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20954cd-4950-4b1e-ba3f-9a8097e5acb2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:25.953729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:25.954398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.954717 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.954993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.955745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:25.957476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:25.976339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1269/2538] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:25.980586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:25.991068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8f04c-907a-4bfb-83e7-ead5a8bdc118 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:25.995615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cc96b-d45e-46df-afb4-3265a83a7472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:25.996279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cc96b-d45e-46df-afb4-3265a83a7472 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:25.996559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cc96b-d45e-46df-afb4-3265a83a7472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:25.996828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cc96b-d45e-46df-afb4-3265a83a7472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:25.997520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8cc96b-d45e-46df-afb4-3265a83a7472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:25.999139 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8cc96b-d45e-46df-afb4-3265a83a7472 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:26.010285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1270/2539] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:19:26.014568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:26.024658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd415ff-3a51-46c9-8df1-a243d441f13f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:26.029179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:26.029837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:26.030116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:26.030389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] PATH_INFO: `/v3/projects/3927b190cb93485ebc7a74c55493f778/users/79e6819df98f4b0ea9c52e784f0f2b67/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:26.046920 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3927b190cb93485ebc7a74c55493f778, user_id=79e6819df98f4b0ea9c52e784f0f2b67, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:26.049716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:26.071979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1270/2540] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:26 2022] PUT /identity/v3/projects/3927b190cb93485ebc7a74c55493f778/users/79e6819df98f4b0ea9c52e784f0f2b67/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:26.073672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a4cd6-8e8b-40ae-8e5c-8a6beacf45fb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:26.073967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a4cd6-8e8b-40ae-8e5c-8a6beacf45fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:26.074245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a4cd6-8e8b-40ae-8e5c-8a6beacf45fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:26.093595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ec5a4cd6-8e8b-40ae-8e5c-8a6beacf45fb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:26.097392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ec5a4cd6-8e8b-40ae-8e5c-8a6beacf45fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:26.159290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1271/2541] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:26.169756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:26.171559 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:26.237691 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046544a8-4749-4bd4-a88c-5c29cb590e5a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:26.241303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:26.241933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:26.242246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:26.242487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:26.252642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:26.254414 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:26.268486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1271/2542] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:27.158671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.159304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.159304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.169273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:27.189963 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] MFA Rules not processed for user `79e6819df98f4b0ea9c52e784f0f2b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:27.191225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:27.261382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1272/2543] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:27.266997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:27.279881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dc4f5-0bad-48d1-b620-64bafe61f7a5 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:27.285442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:27.286640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.286983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.287315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.288088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:27.290045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:27.323795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1272/2544] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:27 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:27.328029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:27.339134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b1e10-2575-41d0-b36f-19d0c185e8ca None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:27.399226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:27.399818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.400113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.400396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.401063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:27.402992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:27.426893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1273/2545] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:27 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:27.431074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:27.441608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fc363-814b-492c-9b86-5b5d8e822bbb admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:27.446386 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:27.447006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.447291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.447568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.448304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:27.449910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:27.460825 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1273/2546] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:27.465385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:27.477984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca91962-f723-4711-a52a-09528ca914bc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:27.485191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:27.486176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.486609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.487022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] PATH_INFO: `/v3/projects/82ad5c8dbd3a46ab976ccf448fc4178e/users/7beda7c95ba8421e843f84d82e9c2de9/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.508972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ad5c8dbd3a46ab976ccf448fc4178e, user_id=7beda7c95ba8421e843f84d82e9c2de9, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:27.512495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:27.533478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1274/2547] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:27 2022] PUT /identity/v3/projects/82ad5c8dbd3a46ab976ccf448fc4178e/users/7beda7c95ba8421e843f84d82e9c2de9/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:27.537978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:27.549042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d2b4-d5de-4004-b33b-fed64f01aabf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:27.595967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:27.613708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b04cb4-3a0d-4f40-9555-8b5c285114b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:27.614437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b04cb4-3a0d-4f40-9555-8b5c285114b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.614680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b04cb4-3a0d-4f40-9555-8b5c285114b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.614964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b04cb4-3a0d-4f40-9555-8b5c285114b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.615672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b04cb4-3a0d-4f40-9555-8b5c285114b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:27.617305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b04cb4-3a0d-4f40-9555-8b5c285114b8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:27.627989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1274/2548] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:27.632365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:27.643086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05592d31-6259-4d2b-8d7b-96c1ae5a925b admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:27.647954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:27.648608 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:27.648896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:27.649163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] PATH_INFO: `/v3/projects/82ad5c8dbd3a46ab976ccf448fc4178e/users/7beda7c95ba8421e843f84d82e9c2de9/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:27.651285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ad5c8dbd3a46ab976ccf448fc4178e, user_id=7beda7c95ba8421e843f84d82e9c2de9, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:27.653929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:27.665030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1275/2549] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:27 2022] PUT /identity/v3/projects/82ad5c8dbd3a46ab976ccf448fc4178e/users/7beda7c95ba8421e843f84d82e9c2de9/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:28.385945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:28.386445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:28.386547 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:28.416531 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] MFA Rules not processed for user `7beda7c95ba8421e843f84d82e9c2de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:28.417733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:28.497915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1275/2550] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:29.695891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:29.745902 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:29.762131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43982-3b0a-45b8-8bf8-3717ac90a092 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:29.767001 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:29.767647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:29.767958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:29.768238 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] PATH_INFO: `/v3/users/79e6819df98f4b0ea9c52e784f0f2b67` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:29.769464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e6819df98f4b0ea9c52e784f0f2b67)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:29.771314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:29.820404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e6819df98f4b0ea9c52e784f0f2b67'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:29.821441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e6819df98f4b0ea9c52e784f0f2b67'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:29.829315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e6819df98f4b0ea9c52e784f0f2b67'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:29.830241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e6819df98f4b0ea9c52e784f0f2b67'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:29.839494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e6819df98f4b0ea9c52e784f0f2b67'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:29.858130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1276/2551] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:29 2022] DELETE /identity/v3/users/79e6819df98f4b0ea9c52e784f0f2b67 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:30.057174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:30.070886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0eb18-5d97-4c76-8bcc-24313047c031 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:30.136812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:30.137745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.138025 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.138304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] PATH_INFO: `/v3/projects/3927b190cb93485ebc7a74c55493f778` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.139472 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3927b190cb93485ebc7a74c55493f778)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:30.141422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:30.189251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3927b190cb93485ebc7a74c55493f778', 'request_id': 'req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.195086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3927b190cb93485ebc7a74c55493f778', 'request_id': 'req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.201409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3927b190cb93485ebc7a74c55493f778', 'request_id': 'req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.205099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:30.216667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] The token cache is being invalidate because project 3927b190cb93485ebc7a74c55493f778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:30.217884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.219073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.220732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1276/2552] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:30 2022] DELETE /identity/v3/projects/3927b190cb93485ebc7a74c55493f778 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:30.224773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:30.226265 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:30.240886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e92c0-9b42-4958-93e3-0c667e9d5a6d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:30.307803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:30.308477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.308763 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.309039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] PATH_INFO: `/v3/users/7beda7c95ba8421e843f84d82e9c2de9` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.310266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7beda7c95ba8421e843f84d82e9c2de9)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:30.312254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:30.380059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7beda7c95ba8421e843f84d82e9c2de9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.380815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7beda7c95ba8421e843f84d82e9c2de9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.390012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7beda7c95ba8421e843f84d82e9c2de9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.391170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7beda7c95ba8421e843f84d82e9c2de9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.401046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4cefb777-c267-4313-a08a-8339c4395136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7beda7c95ba8421e843f84d82e9c2de9'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.408088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:30.429773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1277/2553] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:30 2022] DELETE /identity/v3/users/7beda7c95ba8421e843f84d82e9c2de9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:30.613334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:30.614779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:30.628622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea6d05-83d6-47ae-8e36-aa8c816d6d2b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:30.690131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:30.690791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.691068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.691324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] PATH_INFO: `/v3/projects/82ad5c8dbd3a46ab976ccf448fc4178e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.692467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ad5c8dbd3a46ab976ccf448fc4178e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:30.694385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:30.743127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ad5c8dbd3a46ab976ccf448fc4178e', 'request_id': 'req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.749111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ad5c8dbd3a46ab976ccf448fc4178e', 'request_id': 'req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.755718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ad5c8dbd3a46ab976ccf448fc4178e', 'request_id': 'req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.772299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] The token cache is being invalidate because project 82ad5c8dbd3a46ab976ccf448fc4178e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:30.772588 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.773212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-17eb799c-d4ef-43c2-b19f-0271f8e97cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:30.774977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1277/2554] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:30 2022] DELETE /identity/v3/projects/82ad5c8dbd3a46ab976ccf448fc4178e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:30.779994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.780487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.780582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.800664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:30.801865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:30.867150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1278/2555] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:30.875146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.875430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.875704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.896019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:30.897234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:30.905673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1278/2556] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:30.910054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:30.920576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78853b5-aec0-4c3a-94c6-efee301aa62c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:30.926156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:30.926793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.927069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.927339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.928063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:19:30.928373 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:30.930028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52c21-a86c-413d-ae31-0c9c3e6ce34e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:30.943509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1279/2557] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:19:30.948085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:30.953913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.954191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.954511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.958043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c280d-5703-4df4-bcec-ffc99db74225 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:30.964698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:30.965339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:30.965624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:30.965905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:30.966574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:30.968505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:30.974448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] MFA Rules not processed for user `2637eface78b4813b91967ce66a59c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:30.975666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:30.997227 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1279/2558] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:30 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:31.001453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.043475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1280/2559] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:31.062091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:31.066965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee6e7f-3741-4588-97e6-45bcaeaf8095 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.073939 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.075051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.075681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.076238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.077276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.079242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.104990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1280/2560] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:31 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:31.106242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.117232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510949f-98c9-4501-9be0-37f4e0e905be None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.121814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.122513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.122860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.123224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.124027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.125707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.136164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1281/2561] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:31.140390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.151304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970d1a6-de36-41cb-89fe-e6f668bbbcdd admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.156615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82b25-bc29-45cb-845c-eec16f1331ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.157312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82b25-bc29-45cb-845c-eec16f1331ac admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.157662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82b25-bc29-45cb-845c-eec16f1331ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.158008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82b25-bc29-45cb-845c-eec16f1331ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.158776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b82b25-bc29-45cb-845c-eec16f1331ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.160485 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b82b25-bc29-45cb-845c-eec16f1331ac admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.172774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1281/2562] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:31.176098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.186967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8bbb3-b672-47e9-a3ea-5c0991e6a94d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.191871 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.192517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.192794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.193066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] PATH_INFO: `/v3/projects/ec50ab44d2fc4abaa6f03b2b966f0046/users/d39ed04af3c8485a85f4023c85f56ba5/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.207620 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec50ab44d2fc4abaa6f03b2b966f0046, user_id=d39ed04af3c8485a85f4023c85f56ba5, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.210290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.224208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1282/2563] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:31 2022] PUT /identity/v3/projects/ec50ab44d2fc4abaa6f03b2b966f0046/users/d39ed04af3c8485a85f4023c85f56ba5/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:31.236944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.237102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.237314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.265499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] MFA Rules not processed for user `d39ed04af3c8485a85f4023c85f56ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:31.266720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:31.335731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1282/2564] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:31.340323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.357748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.359146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:31.388683 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:31.408099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6479f12-69c9-451c-8ade-54c32f8623ca admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.413060 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.413687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.413968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.414244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.414951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.416903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.417169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59753cf-176c-4b79-8bfb-158ef12f9d21 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.420757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.421384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.421654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.421929 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.423900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:31.433627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.435288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.446104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1283/2565] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:31 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:31.450316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.451887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1283/2566] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:19:31.544580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0819-d3a5-439a-ad57-c312fa21c68c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.551121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.552357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.552741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.553115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.553894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.556069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.586322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1284/2567] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:31 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:31.591321 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.605166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656139a0-1174-49e9-a703-4b85a40be49d service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.610263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.610936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.611214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.611887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.612264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.614145 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.629257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1284/2568] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:31.630561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.641270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678d02a-c1ff-4b03-9bfe-84a65eee300f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.648451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.649121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.649439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.649682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] PATH_INFO: `/v3/projects/64aaa4b2bcee46849de737683b619e91/users/696a0e27d6d94b3694cf761d6aed3de7/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.670431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64aaa4b2bcee46849de737683b619e91, user_id=696a0e27d6d94b3694cf761d6aed3de7, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.675407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.692481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1285/2569] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:31 2022] PUT /identity/v3/projects/64aaa4b2bcee46849de737683b619e91/users/696a0e27d6d94b3694cf761d6aed3de7/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:31.696317 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.706422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba206f-10d6-4f9c-a29d-d172df9487ff admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.769265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:31.776562 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fc51a-479e-4862-96a5-be1fd574169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.777198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fc51a-479e-4862-96a5-be1fd574169d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.777817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fc51a-479e-4862-96a5-be1fd574169d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.778174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fc51a-479e-4862-96a5-be1fd574169d admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.778965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fc51a-479e-4862-96a5-be1fd574169d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.780671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fc51a-479e-4862-96a5-be1fd574169d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.791357 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1285/2570] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:31.800115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:31.812825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05503f17-edf9-433d-983c-e48a733f7a86 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:31.818258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:31.819456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.819742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.820045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] PATH_INFO: `/v3/projects/64aaa4b2bcee46849de737683b619e91/users/696a0e27d6d94b3694cf761d6aed3de7/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.825376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64aaa4b2bcee46849de737683b619e91, user_id=696a0e27d6d94b3694cf761d6aed3de7, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:31.828095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:31.840000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1286/2571] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:31 2022] PUT /identity/v3/projects/64aaa4b2bcee46849de737683b619e91/users/696a0e27d6d94b3694cf761d6aed3de7/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:31.850681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:31.851120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:31.851519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:31.901102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] MFA Rules not processed for user `696a0e27d6d94b3694cf761d6aed3de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:31.902383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:31.983180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1286/2572] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:32.005726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:32.007811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:32.081084 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91173da1-beea-482c-94d7-c3dcb984c19c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:32.088057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2bd0-67d3-406f-b2af-99a513909e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:32.088702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2bd0-67d3-406f-b2af-99a513909e2f service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:32.089003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2bd0-67d3-406f-b2af-99a513909e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:32.089258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2bd0-67d3-406f-b2af-99a513909e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:32.101632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece2bd0-67d3-406f-b2af-99a513909e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:32.103226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece2bd0-67d3-406f-b2af-99a513909e2f service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:32.116491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1287/2573] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:32.276066 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:32.338499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ff5f-bab1-4538-8501-8e82f87cd9a5 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:32.342937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:32.343580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:32.343902 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:32.344175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] PATH_INFO: `/v3/users/d39ed04af3c8485a85f4023c85f56ba5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:32.345413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39ed04af3c8485a85f4023c85f56ba5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:32.347371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:32.400092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39ed04af3c8485a85f4023c85f56ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:32.400820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39ed04af3c8485a85f4023c85f56ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:32.407865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39ed04af3c8485a85f4023c85f56ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:32.408467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39ed04af3c8485a85f4023c85f56ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:32.417666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39ed04af3c8485a85f4023c85f56ba5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:32.430699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:32.436347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1287/2574] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:32 2022] DELETE /identity/v3/users/d39ed04af3c8485a85f4023c85f56ba5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:32.440987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:32.441276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:32.441558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:32.460263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:32.461630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:32.478343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:32.524672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1288/2575] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:32.534663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:32.536590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:32.594775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3af38-c872-478a-95c2-2a1284ee62c4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:32.598708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:32.599328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:32.599604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:32.599903 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:32.609442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:32.610950 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:32.620586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1288/2576] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:32.911201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:32.924003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:32.940852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562f00f-08ee-431e-86dd-8c33a3eb61d8 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:32.946068 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:32.947207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:32.947862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:32.948303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] PATH_INFO: `/v3/projects/ec50ab44d2fc4abaa6f03b2b966f0046` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:32.949698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec50ab44d2fc4abaa6f03b2b966f0046)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:32.951668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:33.002252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec50ab44d2fc4abaa6f03b2b966f0046', 'request_id': 'req-1a334107-890b-404d-8f14-dbaa52c749fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.011659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec50ab44d2fc4abaa6f03b2b966f0046', 'request_id': 'req-1a334107-890b-404d-8f14-dbaa52c749fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.022984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec50ab44d2fc4abaa6f03b2b966f0046', 'request_id': 'req-1a334107-890b-404d-8f14-dbaa52c749fb'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.037366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] The token cache is being invalidate because project ec50ab44d2fc4abaa6f03b2b966f0046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:33.037652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.038264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-1a334107-890b-404d-8f14-dbaa52c749fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.039800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1289/2577] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:32 2022] DELETE /identity/v3/projects/ec50ab44d2fc4abaa6f03b2b966f0046 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:33.043747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:33.053535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad1c05-89a3-40ef-9ea9-20afcc4d6b7e service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:33.111000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:33.111710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.112031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.112299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] PATH_INFO: `/v3/users/696a0e27d6d94b3694cf761d6aed3de7` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.114018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696a0e27d6d94b3694cf761d6aed3de7)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:33.116000 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:33.176830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696a0e27d6d94b3694cf761d6aed3de7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.177563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696a0e27d6d94b3694cf761d6aed3de7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.184978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696a0e27d6d94b3694cf761d6aed3de7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.185612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696a0e27d6d94b3694cf761d6aed3de7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.195810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696a0e27d6d94b3694cf761d6aed3de7'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.217050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1289/2578] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:33 2022] DELETE /identity/v3/users/696a0e27d6d94b3694cf761d6aed3de7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:33.310402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.310915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.311382 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.337891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:33.339211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:33.397615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1290/2579] 10.222.0.30 () {58 vars in 967 bytes} [Fri Aug 5 05:19:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:33.410615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:33.412543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:33.433931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:33.483418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edea7f-693c-47d7-b477-f2533683bcd5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:33.488646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fc81b-10a7-482a-aac7-cb2968f1e794 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:33.489275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc81b-10a7-482a-aac7-cb2968f1e794 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.489555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc81b-10a7-482a-aac7-cb2968f1e794 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.489830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fc81b-10a7-482a-aac7-cb2968f1e794 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.499620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fc81b-10a7-482a-aac7-cb2968f1e794 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:33.501542 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fc81b-10a7-482a-aac7-cb2968f1e794 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:33.508094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:33.509622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:33.511326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1290/2580] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:19:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:19:33.584134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:33.601028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d15da-8b9b-4ecc-94c3-ba8fab3f09fd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:33.606738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:33.608213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.608507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.608788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] PATH_INFO: `/v3/projects/64aaa4b2bcee46849de737683b619e91` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.609988 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64aaa4b2bcee46849de737683b619e91)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:33.612008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:33.662405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64aaa4b2bcee46849de737683b619e91', 'request_id': 'req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.670247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64aaa4b2bcee46849de737683b619e91', 'request_id': 'req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.677805 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64aaa4b2bcee46849de737683b619e91', 'request_id': 'req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.690967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] The token cache is being invalidate because project 64aaa4b2bcee46849de737683b619e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:33.691262 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.691937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-30e9debd-c3a2-46e2-ac66-1b63c03e9ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:33.693590 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1291/2581] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:33 2022] DELETE /identity/v3/projects/64aaa4b2bcee46849de737683b619e91 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:33.698673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.699031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.699368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.720068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:33.721234 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:33.800379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1291/2582] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:33.808649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.809164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.809595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.845308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:33.848459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:33.859982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1292/2583] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:33.864520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:33.877643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651e348-83f5-4ad2-8f66-f63f9b9c12df None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:33.882621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:33.883724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.883724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.883965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.884685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:19:33.885021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:33.886723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:33.905420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1292/2584] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:19:33.905997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:33.916249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797f51d-9e47-4d9a-bcd1-ae73cef93bae None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:33.922099 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:33.922822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:33.923219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:33.923588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:33.924678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:33.926694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:33.963662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1293/2585] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:33 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:19:33.969038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:33.979533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52a57-c7c0-4679-9335-d2b1fa98e50d admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.045909 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.046050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.046251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.046525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.047184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.049686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.073715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1293/2586] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:33 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:34.078116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.088951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955e3e1-77db-4d63-9c02-f1eae004da05 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.093760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.094388 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.094655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.094924 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.095836 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.097469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.107641 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1294/2587] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:19:34.114629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.131096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a74dd-babf-4bfa-9c4b-2df2fd15bd00 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.149246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc2590-89f7-4f86-a341-9fe09605fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.150031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc2590-89f7-4f86-a341-9fe09605fa62 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.150364 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc2590-89f7-4f86-a341-9fe09605fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.150690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc2590-89f7-4f86-a341-9fe09605fa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.151399 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc2590-89f7-4f86-a341-9fe09605fa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.153901 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc2590-89f7-4f86-a341-9fe09605fa62 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.166168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1294/2588] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:34.170530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.180770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373734-2a6d-44b1-b04d-7aec709d889c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.186427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.187090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.187407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.188034 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] PATH_INFO: `/v3/projects/5d5431990c8441f7b32c79b57e9843a3/users/5f134242d92d449487f4527a47777463/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.213436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5431990c8441f7b32c79b57e9843a3, user_id=5f134242d92d449487f4527a47777463, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.216331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.231441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1295/2589] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:34 2022] PUT /identity/v3/projects/5d5431990c8441f7b32c79b57e9843a3/users/5f134242d92d449487f4527a47777463/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:19:34.245420 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.245785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.246107 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.277779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] MFA Rules not processed for user `5f134242d92d449487f4527a47777463`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:34.278919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:34.282968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:34.346188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1295/2590] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:34.350654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.414537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541a1f5-b059-4c84-a388-4c7a514001ce admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.419427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.420123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.420458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.420811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.421500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.423458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.450490 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1296/2591] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:34 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:34.455056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.466586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266c65e-0c30-4665-8182-6fc3556979df None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.524726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.525545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.525826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.526122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.526810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.528982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.555031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1296/2592] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:34 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:34.556398 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.561847 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:34.567140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b072-2edc-4754-bfea-35c17f0b17ac admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.573759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.574451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.575190 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.575526 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.576472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.578112 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.588933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1297/2593] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:34.593857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.604505 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc462f-1503-45b4-9b1f-7de547d68db4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.609184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.609861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.610139 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.610418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] PATH_INFO: `/v3/projects/470bd720f8e5460abf3a3f1a24a95b6f/users/bbda4aab672647e68dddcee3999ed3e0/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.625330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470bd720f8e5460abf3a3f1a24a95b6f, user_id=bbda4aab672647e68dddcee3999ed3e0, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.628120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.643709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1297/2594] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:34 2022] PUT /identity/v3/projects/470bd720f8e5460abf3a3f1a24a95b6f/users/bbda4aab672647e68dddcee3999ed3e0/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:34.647877 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.658740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92da1be-3c0a-471b-8fba-48e7d26d23dd admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.732966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59751062-469b-4c6d-a6c0-fa6a6eb46193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.733739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59751062-469b-4c6d-a6c0-fa6a6eb46193 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.734086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59751062-469b-4c6d-a6c0-fa6a6eb46193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.734424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59751062-469b-4c6d-a6c0-fa6a6eb46193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.735195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59751062-469b-4c6d-a6c0-fa6a6eb46193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.737180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59751062-469b-4c6d-a6c0-fa6a6eb46193 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.750098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1298/2595] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:34.754293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.765410 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d9f3d-3568-410e-82da-3c6f65d8db40 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.770737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.771426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.771707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.772393 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] PATH_INFO: `/v3/projects/470bd720f8e5460abf3a3f1a24a95b6f/users/bbda4aab672647e68dddcee3999ed3e0/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.774001 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=470bd720f8e5460abf3a3f1a24a95b6f, user_id=bbda4aab672647e68dddcee3999ed3e0, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:34.776667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:34.797608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1298/2596] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:34 2022] PUT /identity/v3/projects/470bd720f8e5460abf3a3f1a24a95b6f/users/bbda4aab672647e68dddcee3999ed3e0/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:34.800416 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.800716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.800998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.833475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] MFA Rules not processed for user `bbda4aab672647e68dddcee3999ed3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:34.834927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:34.913978 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1299/2597] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:34.925514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:34.927320 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:34.984822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45acd94-2115-4481-b2b9-a41a9ecde83f admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:34.988674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:34.989329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:34.989614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:34.989877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:34.999484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:35.001015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:35.006355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:35.014298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1299/2598] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:35.098607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:35.108971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87436d8e-3b9a-4c84-8afa-37ad4eaad76d None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:35.112783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69765ea5-7def-4f1e-a040-26586719e75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:35.113427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69765ea5-7def-4f1e-a040-26586719e75c service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:35.113687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69765ea5-7def-4f1e-a040-26586719e75c service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:35.113962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69765ea5-7def-4f1e-a040-26586719e75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:35.189806 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69765ea5-7def-4f1e-a040-26586719e75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:35.191896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69765ea5-7def-4f1e-a040-26586719e75c service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:35.203007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1300/2599] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:19:40.049232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:40.121426 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e86c2d-ab58-4d5b-8759-29551fb3e266 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:40.126050 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:40.126727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:40.127133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:40.127372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] PATH_INFO: `/v3/users/5f134242d92d449487f4527a47777463` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:40.128769 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f134242d92d449487f4527a47777463)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:40.130736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:40.187298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f134242d92d449487f4527a47777463'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.189967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f134242d92d449487f4527a47777463'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.199900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f134242d92d449487f4527a47777463'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.200738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f134242d92d449487f4527a47777463'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.209852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f134242d92d449487f4527a47777463'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.231787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1300/2600] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:40 2022] DELETE /identity/v3/users/5f134242d92d449487f4527a47777463 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:40.233650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:40.234123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:40.234252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:40.253383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:40.254860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:40.261021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:40.315019 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1301/2601] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:40.324866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:40.326272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:40.384156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e2fb0-676b-4302-bffe-570e8fa9a2e7 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:40.387919 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:40.388540 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:40.388828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:40.389096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:40.398389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:40.399822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:40.409194 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1301/2602] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:40.693381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:40.694860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:40.708950 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f273a-6d61-4deb-95a4-bc50f773a38f None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:40.714456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:40.715119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:40.715822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:40.716103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] PATH_INFO: `/v3/projects/5d5431990c8441f7b32c79b57e9843a3` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:40.717372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5431990c8441f7b32c79b57e9843a3)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:40.719387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:40.768703 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5431990c8441f7b32c79b57e9843a3', 'request_id': 'req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.775195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5431990c8441f7b32c79b57e9843a3', 'request_id': 'req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.781793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5431990c8441f7b32c79b57e9843a3', 'request_id': 'req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.795202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] The token cache is being invalidate because project 5d5431990c8441f7b32c79b57e9843a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:40.795492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.796132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.797623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1302/2603] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:40 2022] DELETE /identity/v3/projects/5d5431990c8441f7b32c79b57e9843a3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:40.801823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:40.811550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0efb9-ea26-4778-a209-6070e66a1af3 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:40.830135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:40.872146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:40.872800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:40.873085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:40.873363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] PATH_INFO: `/v3/users/bbda4aab672647e68dddcee3999ed3e0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:40.874556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbda4aab672647e68dddcee3999ed3e0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:40.876626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:40.925097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbda4aab672647e68dddcee3999ed3e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.925756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbda4aab672647e68dddcee3999ed3e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.932829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbda4aab672647e68dddcee3999ed3e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.933467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbda4aab672647e68dddcee3999ed3e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.941468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-ea04c582-4d49-4657-b40b-68d460f487a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbda4aab672647e68dddcee3999ed3e0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:40.958461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1302/2604] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:40 2022] DELETE /identity/v3/users/bbda4aab672647e68dddcee3999ed3e0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:41.238653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.240444 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:41.253846 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f34d4-0c60-46f4-9dd1-992fa8bcb7c3 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.313236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.313820 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.314100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.314384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] PATH_INFO: `/v3/projects/470bd720f8e5460abf3a3f1a24a95b6f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.315759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=470bd720f8e5460abf3a3f1a24a95b6f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.317807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.357037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:41.368353 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470bd720f8e5460abf3a3f1a24a95b6f', 'request_id': 'req-76518bb8-b335-4b6a-b86b-99da13035eba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:41.374049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470bd720f8e5460abf3a3f1a24a95b6f', 'request_id': 'req-76518bb8-b335-4b6a-b86b-99da13035eba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:41.381028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470bd720f8e5460abf3a3f1a24a95b6f', 'request_id': 'req-76518bb8-b335-4b6a-b86b-99da13035eba'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:41.397435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] The token cache is being invalidate because project 470bd720f8e5460abf3a3f1a24a95b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:41.397726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:41.398460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-76518bb8-b335-4b6a-b86b-99da13035eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:41.400017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1303/2605] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:41 2022] DELETE /identity/v3/projects/470bd720f8e5460abf3a3f1a24a95b6f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:41.405016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.405571 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.405778 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.426431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:41.427639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:41.488220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1303/2606] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:41.497345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.497718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.498079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.518938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:41.520197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:41.529037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1304/2607] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:19:41.533430 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.543629 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340e13-0c20-459c-a2b4-409e09c0a85e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.548469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.549189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.549469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.549726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.550556 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:19:41.550874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.552497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.564545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1304/2608] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:19:41.568940 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.578010 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8959532-cf84-4ff8-8b3c-6e87f959cef0 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.582913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.583508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.583808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.584090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.584770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.586668 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.614649 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1305/2609] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:41 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:41.618874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.630308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce936-6421-40ff-bb3b-6267f3e69a50 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.657895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:41.697612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.698256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.698873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.699424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.699866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.701817 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.726036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1305/2610] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:41 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:41.729936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.740310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787db168-e338-4f75-a786-f097d60fedc5 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.744903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.745515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.745796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.746066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.746762 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.748439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.758945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1306/2611] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:41.763039 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.773173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25665e-fd44-4e61-b413-6b6ac40816a0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.777905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029575d5-9739-433f-841f-04b1e0be946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.778529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029575d5-9739-433f-841f-04b1e0be946f admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.778824 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029575d5-9739-433f-841f-04b1e0be946f admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.779081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029575d5-9739-433f-841f-04b1e0be946f admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.779788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029575d5-9739-433f-841f-04b1e0be946f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.781423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029575d5-9739-433f-841f-04b1e0be946f admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.792945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1306/2612] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:19:41.797290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:41.807570 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cbbde-a3bd-402f-ad7a-a6e3a5daf2e1 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:41.812232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:41.812923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.813207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.813485 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] PATH_INFO: `/v3/projects/62017bc475c8404db579e3012f84df7e/users/bb4a092416b444aab9ecde26413edf1a/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.828178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62017bc475c8404db579e3012f84df7e, user_id=bb4a092416b444aab9ecde26413edf1a, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:41.830809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:41.845209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1307/2613] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:41 2022] PUT /identity/v3/projects/62017bc475c8404db579e3012f84df7e/users/bb4a092416b444aab9ecde26413edf1a/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:19:41.855661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:41.855989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:41.856265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:41.885018 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] MFA Rules not processed for user `bb4a092416b444aab9ecde26413edf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:41.886232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:41.953615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1307/2614] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:41.958279 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.020684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe028f-a33e-4fbf-808e-ed16d8feb3b8 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.025615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.026243 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.026524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.026797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.027467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.029512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.059487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1308/2615] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:41 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:42.063820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.076054 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f5800-a151-4862-9e7e-65df736e894c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.135685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.136256 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.136530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.136803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.137468 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.139371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.166815 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1308/2616] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:42 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:42.171177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.182553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb25ea-19c7-40a6-948a-26cdc9c91cf7 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.187495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.188138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.188419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.188692 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.189387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.191022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.202767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1309/2617] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:42.207166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.217303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded474-fc93-488c-98ae-6e6ae81a1d7c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.222983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.223637 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.223954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.224223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] PATH_INFO: `/v3/projects/9db72731e5c04a22b259d530d9328561/users/c86befcd85d04b5cbfb4992275e3727d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.240372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db72731e5c04a22b259d530d9328561, user_id=c86befcd85d04b5cbfb4992275e3727d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.243010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.259812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1309/2618] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:42 2022] PUT /identity/v3/projects/9db72731e5c04a22b259d530d9328561/users/c86befcd85d04b5cbfb4992275e3727d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:42.263599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.273945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680cb23-9a0e-40e0-b65a-22615eb4e511 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.309254 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:42.335133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572ea33-c81e-4a62-92de-20847794c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.335751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572ea33-c81e-4a62-92de-20847794c02d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.336057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572ea33-c81e-4a62-92de-20847794c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.336331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572ea33-c81e-4a62-92de-20847794c02d admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.337045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572ea33-c81e-4a62-92de-20847794c02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.338650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572ea33-c81e-4a62-92de-20847794c02d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.348782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1310/2619] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:42.354855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.361123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:42.369267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89a2fa-f95e-4292-a2e7-d2125bcc68ba admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.374925 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.375605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.375923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.376223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] PATH_INFO: `/v3/projects/9db72731e5c04a22b259d530d9328561/users/c86befcd85d04b5cbfb4992275e3727d/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.378379 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db72731e5c04a22b259d530d9328561, user_id=c86befcd85d04b5cbfb4992275e3727d, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.381272 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.392593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1310/2620] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:42 2022] PUT /identity/v3/projects/9db72731e5c04a22b259d530d9328561/users/c86befcd85d04b5cbfb4992275e3727d/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:42.403247 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.403559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.403863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.434120 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] MFA Rules not processed for user `c86befcd85d04b5cbfb4992275e3727d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:42.435328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:42.506283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1311/2621] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:42.517177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:42.519172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:42.579286 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c11020-ff61-4f02-81f5-d675bad8a812 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:42.583093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccf2a7-50bb-4597-a755-a5aea30460e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:42.583716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf2a7-50bb-4597-a755-a5aea30460e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:42.584028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf2a7-50bb-4597-a755-a5aea30460e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:42.584303 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccf2a7-50bb-4597-a755-a5aea30460e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:42.593764 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccf2a7-50bb-4597-a755-a5aea30460e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:42.595191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccf2a7-50bb-4597-a755-a5aea30460e5 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:42.604792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1311/2622] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:42.977016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:43.046426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aae99-65c2-46cf-8d7d-f2a164de4d8b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:43.051313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:43.051971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:43.052252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:43.052522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] PATH_INFO: `/v3/users/bb4a092416b444aab9ecde26413edf1a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:43.053716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4a092416b444aab9ecde26413edf1a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:43.055634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:43.108292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4a092416b444aab9ecde26413edf1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.108947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4a092416b444aab9ecde26413edf1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.115631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4a092416b444aab9ecde26413edf1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.116344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4a092416b444aab9ecde26413edf1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.118872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:43.125026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4a092416b444aab9ecde26413edf1a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.144063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1312/2623] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:42 2022] DELETE /identity/v3/users/bb4a092416b444aab9ecde26413edf1a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:43.148894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:43.149195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:43.149539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:43.169129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:43.170343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:43.230799 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1312/2624] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:43.240733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:43.242201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:43.303971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f213b9-76db-4a33-a22b-e3e20be57ded admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:43.308633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:43.309259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:43.309554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:43.309829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:43.320589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:43.322028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:43.332804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1313/2625] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:43.625580 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:43.627394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:43.640308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0865e8a-f614-42fe-98b2-a2280730211f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:43.645979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:43.646654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:43.647113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:43.647610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] PATH_INFO: `/v3/projects/62017bc475c8404db579e3012f84df7e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:43.648945 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62017bc475c8404db579e3012f84df7e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:43.650874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:43.706277 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62017bc475c8404db579e3012f84df7e', 'request_id': 'req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.713051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62017bc475c8404db579e3012f84df7e', 'request_id': 'req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.719844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62017bc475c8404db579e3012f84df7e', 'request_id': 'req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.736016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] The token cache is being invalidate because project 62017bc475c8404db579e3012f84df7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:43.736310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.737041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.738651 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1313/2626] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:43 2022] DELETE /identity/v3/projects/62017bc475c8404db579e3012f84df7e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:43.742866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:43.754002 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0fd55-c079-4f9d-a015-bb342158b3f2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:43.812656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:43.813705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:43.814053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:43.814449 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] PATH_INFO: `/v3/users/c86befcd85d04b5cbfb4992275e3727d` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:43.815879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86befcd85d04b5cbfb4992275e3727d)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:43.818632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:43.876159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86befcd85d04b5cbfb4992275e3727d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.876824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86befcd85d04b5cbfb4992275e3727d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.884146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86befcd85d04b5cbfb4992275e3727d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.884785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86befcd85d04b5cbfb4992275e3727d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.893285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7955c823-0dad-4e74-8f8c-d9537da44a52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86befcd85d04b5cbfb4992275e3727d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:43.909364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:43.917553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1314/2627] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:43 2022] DELETE /identity/v3/users/c86befcd85d04b5cbfb4992275e3727d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:44.196560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.198129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:44.215782 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524fcb5-d191-44c1-82a2-9e0d9e81fbb5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.230819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:44.277484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.278133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.278403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.278677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] PATH_INFO: `/v3/projects/9db72731e5c04a22b259d530d9328561` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.279852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db72731e5c04a22b259d530d9328561)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.281755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.333184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db72731e5c04a22b259d530d9328561', 'request_id': 'req-292180fa-303d-49d3-bff5-74c4860f89d1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:44.339324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db72731e5c04a22b259d530d9328561', 'request_id': 'req-292180fa-303d-49d3-bff5-74c4860f89d1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:44.345962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db72731e5c04a22b259d530d9328561', 'request_id': 'req-292180fa-303d-49d3-bff5-74c4860f89d1'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:44.360019 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] The token cache is being invalidate because project 9db72731e5c04a22b259d530d9328561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:44.360310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:44.361105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-292180fa-303d-49d3-bff5-74c4860f89d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:44.362998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1314/2628] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:44 2022] DELETE /identity/v3/projects/9db72731e5c04a22b259d530d9328561 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:44.369197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.369713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.369933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.389694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:44.390993 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:44.455619 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1315/2629] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:44.463985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.464450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.464899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.489106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:44.490446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:44.503169 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1315/2630] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:44.504602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.514596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15226-2179-4c90-b9c9-71aafa3baab7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.519372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.520014 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.520295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.520567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.521263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:19:44.521572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.523419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.535583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1316/2631] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:19:44.540518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.550191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644650-0156-44eb-b04b-d011fc5592ef None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.555949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.556564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.556858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.557132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.557808 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.559715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.588258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1316/2632] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:44 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:19:44.594092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.604423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad37f4e-75b9-4e4e-8353-38f48f2fbaa4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.661797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.662429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.662711 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.662995 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.663687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.665639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.689047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1317/2633] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:44 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:19:44.693695 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.704725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3e0c2-eba6-41af-bb14-34ed3b9877b3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.710210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.710935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.711305 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.711655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.712398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.714024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.726900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1317/2634] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:44.728178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.738594 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa087de-d71a-44d3-a036-5c31df9bf5f2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.743169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac0ecf-10f3-448b-8355-6c4f4aad78ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.743833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac0ecf-10f3-448b-8355-6c4f4aad78ac admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.744115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac0ecf-10f3-448b-8355-6c4f4aad78ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.744384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac0ecf-10f3-448b-8355-6c4f4aad78ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.745086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac0ecf-10f3-448b-8355-6c4f4aad78ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.746648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac0ecf-10f3-448b-8355-6c4f4aad78ac admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.759507 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1318/2635] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:44.763753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.774199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d4a01-70f9-4854-b614-9b564e2cc530 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:44.778912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:44.779618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.779937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.780211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] PATH_INFO: `/v3/projects/c53805649e244a79ac833486eae22293/users/271079a663254f97af95a202e6c109b1/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.795848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53805649e244a79ac833486eae22293, user_id=271079a663254f97af95a202e6c109b1, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:44.798557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:44.814786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1318/2636] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:44 2022] PUT /identity/v3/projects/c53805649e244a79ac833486eae22293/users/271079a663254f97af95a202e6c109b1/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:44.825748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:44.826090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:44.826314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:44.857983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] MFA Rules not processed for user `271079a663254f97af95a202e6c109b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:44.859313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:44.919585 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:44.931579 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1319/2637] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:44.943754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:44.945965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:45.000429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:45.018644 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fe0a8-6ba3-4919-8c90-bfdcc73766cf admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:45.022763 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfceb1-875a-440a-aa54-e609f5516fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:45.023529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfceb1-875a-440a-aa54-e609f5516fab service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:45.023944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfceb1-875a-440a-aa54-e609f5516fab service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:45.024391 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfceb1-875a-440a-aa54-e609f5516fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:45.034144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfceb1-875a-440a-aa54-e609f5516fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:45.036157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfceb1-875a-440a-aa54-e609f5516fab service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:45.046672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1319/2638] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:47.669264 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:47.734226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e8f6e-937a-42b4-bec6-c979bb23d1df None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:47.739314 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:47.739984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:47.740257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:47.740528 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] PATH_INFO: `/v3/users/271079a663254f97af95a202e6c109b1` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:47.741733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=271079a663254f97af95a202e6c109b1)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:47.743653 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:47.798659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271079a663254f97af95a202e6c109b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:47.799355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271079a663254f97af95a202e6c109b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:47.806517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271079a663254f97af95a202e6c109b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:47.807161 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271079a663254f97af95a202e6c109b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:47.815638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271079a663254f97af95a202e6c109b1'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:47.834420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1320/2639] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:19:47 2022] DELETE /identity/v3/users/271079a663254f97af95a202e6c109b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:47.839297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:47.839828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:47.839961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:47.861432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:47.862878 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:47.928768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1320/2640] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:47.938863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:47.940392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:47.999045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a810f-6900-4cd6-a3c0-f8b8808541fc admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.003399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b992-6e9a-4d9f-9a12-9da542c921a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.004071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b992-6e9a-4d9f-9a12-9da542c921a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.004369 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b992-6e9a-4d9f-9a12-9da542c921a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.004658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b992-6e9a-4d9f-9a12-9da542c921a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.014451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b992-6e9a-4d9f-9a12-9da542c921a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.015926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b992-6e9a-4d9f-9a12-9da542c921a7 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.025932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1321/2641] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:48.300188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.301664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:48.315222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7039d6-5855-4bd0-ac1d-5fffe45959b7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.320496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.321388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.323027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.323696 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] PATH_INFO: `/v3/projects/c53805649e244a79ac833486eae22293` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.325185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53805649e244a79ac833486eae22293)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.327487 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.379168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53805649e244a79ac833486eae22293', 'request_id': 'req-21b51bbd-b50c-422b-9b1d-db572854f4ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:48.384647 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53805649e244a79ac833486eae22293', 'request_id': 'req-21b51bbd-b50c-422b-9b1d-db572854f4ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:48.390896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53805649e244a79ac833486eae22293', 'request_id': 'req-21b51bbd-b50c-422b-9b1d-db572854f4ed'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:48.403578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] The token cache is being invalidate because project c53805649e244a79ac833486eae22293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:19:48.403906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:48.404520 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-21b51bbd-b50c-422b-9b1d-db572854f4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:19:48.406034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1321/2642] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:19:48 2022] DELETE /identity/v3/projects/c53805649e244a79ac833486eae22293 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:48.411445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.411808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.412122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.433878 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:48.435113 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:48.446483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:48.508863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1322/2643] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:48.518631 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.518912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.519223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.539635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:48.540885 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:48.549451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1322/2644] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:48.553775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.564613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5245b-d660-4025-9e6d-6a2d29758e60 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.570639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.571269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.571549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.571857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.572559 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:19:48.572868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.574487 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.587098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1323/2645] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:19:48.591450 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.600985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872984ea-8550-47f6-b9db-de81ee8a92a2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.606783 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.607652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.607652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.607973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.608655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.610590 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.642396 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1323/2646] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:19:48 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:19:48.646743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.656916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49995180-aed9-4ad0-98ae-e4319e7d4499 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.721959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.722631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.722861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.723131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.723829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.725764 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.752418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1324/2647] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:19:48 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:19:48.758433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.769688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4d50d-f01f-4d18-996e-cf17af264fa0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.775301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.776063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.776347 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.776608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.777307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.778915 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.795153 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1324/2648] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:48.799929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.810351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53732e70-bb7b-4e84-959e-c94c08991715 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.815406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211985f6-6d96-4a77-9fc2-e6f7b62f36b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.816041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211985f6-6d96-4a77-9fc2-e6f7b62f36b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.816391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211985f6-6d96-4a77-9fc2-e6f7b62f36b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.816665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211985f6-6d96-4a77-9fc2-e6f7b62f36b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.817367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211985f6-6d96-4a77-9fc2-e6f7b62f36b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.819000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211985f6-6d96-4a77-9fc2-e6f7b62f36b4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.829607 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1325/2649] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:19:48.833696 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.843809 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087e337-c95d-4d16-83ec-816a06bae6a4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:48.849096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:48.849738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.850006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.850314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] PATH_INFO: `/v3/projects/0af046d32c704d8bb13b821da2b8338a/users/06de5afa7d2d4d3aa0c1378ceeb5c4bf/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.866756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af046d32c704d8bb13b821da2b8338a, user_id=06de5afa7d2d4d3aa0c1378ceeb5c4bf, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:48.869506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:48.883623 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1325/2650] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:19:48 2022] PUT /identity/v3/projects/0af046d32c704d8bb13b821da2b8338a/users/06de5afa7d2d4d3aa0c1378ceeb5c4bf/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:19:48.888718 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d15da-0200-499e-aa20-7fbc1ee060b8 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:48.889005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d15da-0200-499e-aa20-7fbc1ee060b8 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:48.889285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d15da-0200-499e-aa20-7fbc1ee060b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:48.908389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-123d15da-0200-499e-aa20-7fbc1ee060b8 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:48.909584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-123d15da-0200-499e-aa20-7fbc1ee060b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:48.970539 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1326/2651] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:48.980632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:48.982446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:48.989313 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:49.052776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c37d8-4813-403e-84d6-6f9664a2b305 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:49.059738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:49.060660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:49.061035 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:49.061397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:49.073336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:49.074777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:49.084538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1326/2652] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:19:49.725914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad828fdf-3cb8-4c0e-8b85-0fbe20eac713 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:49.726464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad828fdf-3cb8-4c0e-8b85-0fbe20eac713 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:49.726464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad828fdf-3cb8-4c0e-8b85-0fbe20eac713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:49.755961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-ad828fdf-3cb8-4c0e-8b85-0fbe20eac713 None None] MFA Rules not processed for user `06de5afa7d2d4d3aa0c1378ceeb5c4bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:49.757189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ad828fdf-3cb8-4c0e-8b85-0fbe20eac713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:49.766775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:19:49.827779 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1327/2653] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:49.838260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:19:49.840082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:49.898681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a16f6-e8c0-42a9-8bcb-a5496761ed1e service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:19:49.902649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:19:49.903266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:49.903544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:49.903806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:49.913480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:19:49.914914 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:19:49.923819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1327/2654] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:19:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:19:51.299925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:19:51.300451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:19:51.300451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:19:51.320716 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] MFA Rules not processed for user `06de5afa7d2d4d3aa0c1378ceeb5c4bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:19:51.321969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:19:51.330562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1328/2655] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:19:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:19:59.919789 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=50 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:01.314998 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=25 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:15.998750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:16.009846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349150fe-e702-4ddd-9afc-0ba0c4115176 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:16.014422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:16.015042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:16.015316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:16.015584 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:16.017672 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:16.030365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:16.031801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:16.041996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1328/2656] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:20:19.666114 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:19.667612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:19.750661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4036-9ccd-439a-a2b7-fc2c00f6122a None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:19.755863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:19.756534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:19.756816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:19.757096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:19.759052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:19.771329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:19.773016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:19.784554 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1329/2657] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:20:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:20:20.016341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.018015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:20.031356 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174bcbb-8c54-4bf6-a291-154b3eab7e18 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.037011 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.037667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.037923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.038199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] PATH_INFO: `/v3/users/b89279618557486f9d5b358b654a906e` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.039437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89279618557486f9d5b358b654a906e)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.041381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.092444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89279618557486f9d5b358b654a906e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.093149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89279618557486f9d5b358b654a906e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.101401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89279618557486f9d5b358b654a906e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.102302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89279618557486f9d5b358b654a906e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.112293 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c395d3c9-99b9-465b-8bba-d37bbd2a4a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89279618557486f9d5b358b654a906e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.137362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1329/2658] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:20:20 2022] DELETE /identity/v3/users/b89279618557486f9d5b358b654a906e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:20:20.321848 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.333955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b3048-868f-402f-8674-62134c6e107e service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.396511 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.397173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.397451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.397686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] PATH_INFO: `/v3/projects/e43f69b71a364a63850d3008309e0c68` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.399052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43f69b71a364a63850d3008309e0c68)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.401045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.453172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43f69b71a364a63850d3008309e0c68', 'request_id': 'req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.459133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43f69b71a364a63850d3008309e0c68', 'request_id': 'req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.465503 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43f69b71a364a63850d3008309e0c68', 'request_id': 'req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.479157 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] The token cache is being invalidate because project e43f69b71a364a63850d3008309e0c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:20:20.479445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.480074 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e415ea0a-f5cf-4170-b9f7-5c64eaa8a4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:20.481588 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1330/2659] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:20:20 2022] DELETE /identity/v3/projects/e43f69b71a364a63850d3008309e0c68 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:20:20.486681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.486969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.487238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.506550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:20.507674 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:20.565828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1330/2660] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:20:20.574775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.575071 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.575343 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.596121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:20.597309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:20.605617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1331/2661] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:20:20.610078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.620264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba602-785b-4c24-b18d-69df8124a7fb None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.625551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae434-fe40-4305-9266-956304251448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.626165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae434-fe40-4305-9266-956304251448 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.626478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae434-fe40-4305-9266-956304251448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.626752 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae434-fe40-4305-9266-956304251448 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.627457 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ae434-fe40-4305-9266-956304251448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:20:20.627788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ae434-fe40-4305-9266-956304251448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.629394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ae434-fe40-4305-9266-956304251448 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.641461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1331/2662] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:20:20.645686 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.655015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df09ed8-d833-4186-ab81-2e4b17f6dbde None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.659931 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.660901 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.661175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.661446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.662109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.664053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.675946 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:20.695041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1332/2663] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:20:20 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:20:20.699958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae434-fe40-4305-9266-956304251448 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.713923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae434-fe40-4305-9266-956304251448 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.760683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:20.775788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.776428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.776741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.777014 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.777682 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.779633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.802593 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1332/2664] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:20:20 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:20:20.806873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.817291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2d995-2bf1-4175-a131-442f9533061e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.822171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.822772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.823052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.823322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.824049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.825727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.835803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1333/2665] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:20:20.839675 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.849860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2b06e-3a53-4127-86e1-f92795d53f06 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.854801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcda18-c966-402e-a602-272e9d245ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.855405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcda18-c966-402e-a602-272e9d245ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.855680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcda18-c966-402e-a602-272e9d245ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.855984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcda18-c966-402e-a602-272e9d245ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.856677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcda18-c966-402e-a602-272e9d245ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.858258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcda18-c966-402e-a602-272e9d245ae5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.868900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1333/2666] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:20:20.873058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:20.883407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295ac19-fd20-488c-b70e-af90044ea654 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:20.888302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:20.888952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.889234 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.889498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] PATH_INFO: `/v3/projects/7ba6b61b39e04bf98627fe6a470ba184/users/7127bedba9fa462fa068c407fe69221a/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.904629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba6b61b39e04bf98627fe6a470ba184, user_id=7127bedba9fa462fa068c407fe69221a, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:20.907336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:20.924832 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1334/2667] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:20:20 2022] PUT /identity/v3/projects/7ba6b61b39e04bf98627fe6a470ba184/users/7127bedba9fa462fa068c407fe69221a/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:20:20.935661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:20.935989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:20.936273 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:20.964955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] MFA Rules not processed for user `7127bedba9fa462fa068c407fe69221a`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:20.966184 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:21.041031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1334/2668] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:20:21.050958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:21.052753 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:21.128456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4ea44-5ba2-4b2b-9a3d-c29d8d333c1e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:21.134869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67739a-0883-4000-92e5-47014b2e1819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:21.135484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67739a-0883-4000-92e5-47014b2e1819 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:21.135741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67739a-0883-4000-92e5-47014b2e1819 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:21.136042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67739a-0883-4000-92e5-47014b2e1819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:21.145447 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67739a-0883-4000-92e5-47014b2e1819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:21.147057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67739a-0883-4000-92e5-47014b2e1819 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:21.156756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1335/2669] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:20:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:20:30.629980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:30.695104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87d52-4423-495a-8eee-9173dc5d7a6b None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:30.700781 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:30.701467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:30.701821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:30.702191 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] PATH_INFO: `/v3/users/7127bedba9fa462fa068c407fe69221a` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:30.704051 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7127bedba9fa462fa068c407fe69221a)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:30.707252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:30.767164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7127bedba9fa462fa068c407fe69221a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:30.767874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7127bedba9fa462fa068c407fe69221a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:30.774800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7127bedba9fa462fa068c407fe69221a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:30.775442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7127bedba9fa462fa068c407fe69221a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:30.777185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:30.784314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7127bedba9fa462fa068c407fe69221a'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:30.802497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1335/2670] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:20:30 2022] DELETE /identity/v3/users/7127bedba9fa462fa068c407fe69221a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:20:30.807218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:30.807827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:30.807827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:30.829057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:30.830819 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:30.890828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1336/2671] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:20:30.901079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:30.902666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:30.964599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5be47-ceb3-46c3-a079-50bb7d0ec658 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:30.968661 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd836324-c7de-4331-8257-7a8f38c31a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:30.969375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836324-c7de-4331-8257-7a8f38c31a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:30.969649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836324-c7de-4331-8257-7a8f38c31a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:30.969921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836324-c7de-4331-8257-7a8f38c31a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:30.979795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd836324-c7de-4331-8257-7a8f38c31a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:30.981242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd836324-c7de-4331-8257-7a8f38c31a42 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:30.991068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1336/2672] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:20:31.273109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.274658 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:31.290378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3674fe-b64b-4b76-837d-1fb6c9260b67 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.297111 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.298481 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.299143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.299424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] PATH_INFO: `/v3/projects/7ba6b61b39e04bf98627fe6a470ba184` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.300752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba6b61b39e04bf98627fe6a470ba184)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.302739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.353626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:31.369740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba6b61b39e04bf98627fe6a470ba184', 'request_id': 'req-c641a91b-4f93-47c7-8a63-9720aa2f6c59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:31.377947 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba6b61b39e04bf98627fe6a470ba184', 'request_id': 'req-c641a91b-4f93-47c7-8a63-9720aa2f6c59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:31.384685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba6b61b39e04bf98627fe6a470ba184', 'request_id': 'req-c641a91b-4f93-47c7-8a63-9720aa2f6c59'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:31.400442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] The token cache is being invalidate because project 7ba6b61b39e04bf98627fe6a470ba184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:20:31.400788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:31.401796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c641a91b-4f93-47c7-8a63-9720aa2f6c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:20:31.404458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1337/2673] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:20:31 2022] DELETE /identity/v3/projects/7ba6b61b39e04bf98627fe6a470ba184 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:20:31.409866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.410594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.410931 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.443641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:31.445195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:31.511719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1337/2674] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:20:31.519715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.520039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.520310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.539823 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:31.541021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:31.551659 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1338/2675] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:20:31.558643 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.569744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528f9ff-d4de-4b44-8233-a8b9bd1f8177 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.575616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.576304 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.576585 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.576864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.577597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:20:31.577908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.579625 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.592344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1338/2676] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:20:31.596689 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.606577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459516f-f93a-47d5-9c85-791bde6527eb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.612175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.612798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.613075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.613345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.614012 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.615938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.647140 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1339/2677] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:20:31 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:20:31.651999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.662873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f23b4a-9ffc-44ab-936a-4f262fbddeb2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.722048 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.722690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.722966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.723237 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.723936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.725918 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.736344 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:31.754953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1339/2678] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:20:31 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:20:31.756185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.766737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e8dab-fc91-4d98-9730-fe3b053e8b5a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.771360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.772098 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.772443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.773230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.773538 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.775221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.786116 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1340/2679] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:20:31.791217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.801239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa704-e3c2-45a1-9121-528aff452f62 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.805898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58716-428d-4416-83ee-ec6d1a3a29c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.806545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58716-428d-4416-83ee-ec6d1a3a29c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.806828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58716-428d-4416-83ee-ec6d1a3a29c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.807105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58716-428d-4416-83ee-ec6d1a3a29c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.808013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c58716-428d-4416-83ee-ec6d1a3a29c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.809627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c58716-428d-4416-83ee-ec6d1a3a29c1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.820753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1340/2680] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:20:31.825031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.835428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a527-23b6-48b8-a41d-78086435bb59 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:31.840088 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:31.840760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.841044 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.841324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] PATH_INFO: `/v3/projects/aa9fdc025617423aa08c7b18df08ce3f/users/bcd8139d51bc480db741185db186242f/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.856138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9fdc025617423aa08c7b18df08ce3f, user_id=bcd8139d51bc480db741185db186242f, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:31.858816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:31.873781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1341/2681] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:20:31 2022] PUT /identity/v3/projects/aa9fdc025617423aa08c7b18df08ce3f/users/bcd8139d51bc480db741185db186242f/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:20:31.885822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:31.886118 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:31.886400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:31.913677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] MFA Rules not processed for user `bcd8139d51bc480db741185db186242f`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:20:31.914906 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:31.981616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1341/2682] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:20:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:20:31.993303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:20:31.995843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:20:32.054042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f6761-6152-4310-b4b6-19892a29c014 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:20:32.057838 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:20:32.058472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:20:32.058752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:20:32.059046 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:20:32.068147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:20:32.069782 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:20:32.079512 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1342/2683] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:20:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:20:41.978159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:20:42.075175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:21:27.533545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:21:27.536102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:21:27.688846 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88e024-aafe-4169-b87d-6d7dbe50a71a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:21:27.694708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:21:27.695339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:21:27.695577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:21:27.695898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:21:27.697856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:21:27.710134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:21:27.711702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:21:27.721962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1342/2684] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:21:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:21:37.717777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=63 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:23:24.603094 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:23:24.604807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:23:24.735269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae229-de79-4880-8123-2cd86c6401f6 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:23:24.741324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:23:24.741966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:23:24.742250 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:23:24.742522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:23:24.744340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:23:24.803103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:23:24.804735 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:23:24.813717 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1343/2685] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:23:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:23:34.809869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=79 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:24:19.669816 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:24:19.671553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:24:19.750619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c589-0b3d-4eac-ae46-12911fdc98ec service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:24:19.754772 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:24:19.755388 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:24:19.755712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:24:19.756065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:24:19.757935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:24:19.768046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:24:19.769639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:24:19.778264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1343/2686] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:24:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:24:29.774519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=35 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:24:50.501519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:24:50.525825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b33e4-164b-4567-8b46-a601ea6fc80d service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:24:50.530534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:24:50.531152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:24:50.531794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:24:50.532096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:24:50.534063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:24:50.594297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:24:50.595740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:24:50.604933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1344/2687] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:24:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:25:00.600861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=32 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:25:32.593959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:25:32.596663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:25:32.631283 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab283c-fa27-450a-a977-f15f3d42b929 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:25:32.635506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:25:32.636158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:25:32.636438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:25:32.636709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:25:32.645461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:25:32.646888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:25:32.656371 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1344/2688] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:25:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:25:42.652117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=11 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:26:35.479125 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:35.491446 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc7f35-819f-4caa-bf7f-595abcf0b7b9 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:35.496589 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:35.497207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:35.497510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:35.497750 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:35.500195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:35.515504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:35.517020 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d26a77-639c-4c1c-b11a-c76b666ac458 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:35.528457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1345/2689] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:26:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:26:40.053656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:40.136831 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e543101-d7ef-4381-a132-5ec6c698b6d0 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:40.141814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:40.142458 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:40.142737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:40.143005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] PATH_INFO: `/v3/users/bcd8139d51bc480db741185db186242f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:40.144196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd8139d51bc480db741185db186242f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:40.146111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:40.196389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd8139d51bc480db741185db186242f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.196964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd8139d51bc480db741185db186242f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.204447 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd8139d51bc480db741185db186242f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.205094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd8139d51bc480db741185db186242f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.216004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd8139d51bc480db741185db186242f'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.236561 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1345/2690] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:26:40 2022] DELETE /identity/v3/users/bcd8139d51bc480db741185db186242f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:26:40.241435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:40.241986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:40.241986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:40.287063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:26:40.288322 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:40.355342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1346/2691] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:26:40.367232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:40.368851 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:40.429281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32fd70-76cb-4de2-993f-5ed28679d2ff admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:40.433168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb9291-ef95-42f7-9d4d-4d99d86ceed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:40.433784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb9291-ef95-42f7-9d4d-4d99d86ceed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:40.434061 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb9291-ef95-42f7-9d4d-4d99d86ceed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:40.434334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb9291-ef95-42f7-9d4d-4d99d86ceed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:40.443440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb9291-ef95-42f7-9d4d-4d99d86ceed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:40.444894 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb9291-ef95-42f7-9d4d-4d99d86ceed9 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:40.454176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1346/2692] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:26:40.735938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:40.737552 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:40.749797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb3e05-962f-4dd2-99d9-c44075523340 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:40.755426 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:40.756135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:40.756418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:40.756691 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] PATH_INFO: `/v3/projects/aa9fdc025617423aa08c7b18df08ce3f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:40.758005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9fdc025617423aa08c7b18df08ce3f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:40.759943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:40.811296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9fdc025617423aa08c7b18df08ce3f', 'request_id': 'req-36c857de-4850-43cf-8639-d8a1e37590a8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.817296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9fdc025617423aa08c7b18df08ce3f', 'request_id': 'req-36c857de-4850-43cf-8639-d8a1e37590a8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.823592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9fdc025617423aa08c7b18df08ce3f', 'request_id': 'req-36c857de-4850-43cf-8639-d8a1e37590a8'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.839006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] The token cache is being invalidate because project aa9fdc025617423aa08c7b18df08ce3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:26:40.839292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.839971 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-36c857de-4850-43cf-8639-d8a1e37590a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:26:40.841524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1347/2693] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:26:40 2022] DELETE /identity/v3/projects/aa9fdc025617423aa08c7b18df08ce3f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:26:40.847353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:40.847663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:40.847976 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:40.853990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:26:40.898508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:26:40.899699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:40.964500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1347/2694] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:26:40.973064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:40.973472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:40.973756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:40.997642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:26:40.997642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:41.005304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1348/2695] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:26:41.009837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.024132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed02e4-650a-493a-8429-6ec859c41f2f None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.029438 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.030271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.031042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.031322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.032079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:26:41.032384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.033970 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.045768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1348/2696] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:26:41.050045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.059532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfb364-07f4-4929-9f13-39ebe83575f7 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.064495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.065107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.065384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.065640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.066310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.068339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.104285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1349/2697] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:26:41 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:26:41.108575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.119096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84987751-f2ca-454b-9fdb-08bb114d77ac admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.178074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.178679 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.178955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.179228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.179922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.181829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.204128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1349/2698] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:26:41 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:26:41.208483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.219613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66aaddd-dd73-42da-acd7-68b5d1a510ff admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.224895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.225500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.225783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.226051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.226730 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.228440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.238418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1350/2699] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:26:41.242497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.252655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ee73a-20c6-460e-a1b4-6c665a5aaace admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.258143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29059-520f-4ee9-9929-12031bb15ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.258741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29059-520f-4ee9-9929-12031bb15ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.259022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29059-520f-4ee9-9929-12031bb15ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.259292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29059-520f-4ee9-9929-12031bb15ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.260023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d29059-520f-4ee9-9929-12031bb15ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.261610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d29059-520f-4ee9-9929-12031bb15ac1 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.275133 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1350/2700] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:26:41.280214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.287024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:26:41.291663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c1198-0e23-4536-a71b-846a41ddac1e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.300126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.301181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.303333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.303650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] PATH_INFO: `/v3/projects/f9abb24676264ae9824782dfa5d92740/users/27eb1354177b48d08fa718ae9bdea996/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.321101 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9abb24676264ae9824782dfa5d92740, user_id=27eb1354177b48d08fa718ae9bdea996, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.323864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.338236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1351/2701] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:26:41 2022] PUT /identity/v3/projects/f9abb24676264ae9824782dfa5d92740/users/27eb1354177b48d08fa718ae9bdea996/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:26:41.353493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.353896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.354291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.386385 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] MFA Rules not processed for user `27eb1354177b48d08fa718ae9bdea996`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:26:41.387616 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:41.423020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:26:41.461827 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1351/2702] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:26:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:26:41.475225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:26:41.477047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:26:41.533202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8ed5e-20ac-41b8-bec6-50ca8974672f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:26:41.537583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449003-0406-413c-b561-d4275019606a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:26:41.538205 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449003-0406-413c-b561-d4275019606a service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:26:41.538484 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449003-0406-413c-b561-d4275019606a service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:26:41.538751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36449003-0406-413c-b561-d4275019606a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:26:41.547865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36449003-0406-413c-b561-d4275019606a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:26:41.549317 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36449003-0406-413c-b561-d4275019606a service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:26:41.558338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1352/2703] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:26:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:26:51.453310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=11 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:26:51.554377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:27:24.560185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:27:24.561958 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:27:24.637060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ee3a6-e13e-4a02-9507-3f923a21d99e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:27:24.642748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:27:24.643403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:27:24.643686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:27:24.643991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:27:24.646108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:27:24.658433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:27:24.660045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:27:24.670015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1352/2704] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:27:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:27:34.665886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=32 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:28:24.889218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449003-0406-413c-b561-d4275019606a service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:28:24.891953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-36449003-0406-413c-b561-d4275019606a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:28:24.912440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36449003-0406-413c-b561-d4275019606a service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:28:24.917171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:28:24.917783 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:28:24.918053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:28:24.918323 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:28:24.920624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:28:25.005744 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:28:25.007288 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:28:25.025839 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1353/2705] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:28:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:28:35.023117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=41 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:29:25.612737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:29:25.614838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:29:25.672389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c9231-87b2-47d1-a765-6fcb190250e5 service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:29:25.676264 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:29:25.676866 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:29:25.677144 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:29:25.677419 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:29:25.679301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:29:25.688719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:29:25.690143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:29:25.697997 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1353/2706] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:29:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:29:35.694723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=26 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:29:51.038029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:29:51.047597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985813-2022-44d4-96be-224d4dd364e9 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:29:51.052063 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:29:51.052681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:29:51.052968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:29:51.053240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:29:51.055207 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:29:51.137864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:29:51.139321 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:29:51.158110 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1354/2707] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:29:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:30:01.155418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=41 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:31:41.196403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:31:41.206680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02bc25-23f5-4279-b5de-5c90f2ba1265 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:31:41.211723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:31:41.212361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:31:41.212639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:31:41.212912 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:31:41.222488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:31:41.223940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:31:41.233310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1354/2708] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:31:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:31:51.229074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:32:29.639011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:32:29.640690 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:32:29.705045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec9dc4-584b-4cc3-9b74-618fa1747e0e service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:32:29.710962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:32:29.711592 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:32:29.711975 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:32:29.712252 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:32:29.721228 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:32:29.722840 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:32:29.732883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1355/2709] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:32:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:32:35.818131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:32:35.828694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b9f2e-8604-4a73-83c3-c05568912b9f service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:32:35.832910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:32:35.833522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:32:35.833797 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:32:35.834219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:32:35.836071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:32:35.909544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:32:35.910982 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:32:35.922280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1355/2710] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:32:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:32:37.114801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:32:37.125042 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1efe2e-fba0-404c-8545-f1654c339f27 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:32:37.129612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:32:37.130219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:32:37.130496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:32:37.130769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:32:37.132937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:32:37.146631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:32:37.148478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:32:37.159499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1356/2711] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:32:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:32:39.789342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:32:39.791147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:32:39.806400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43161446-aca3-4a46-b751-7c3fc760214f service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:32:39.812424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10179938-4d1c-4519-8772-f8255677733c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:32:39.813122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10179938-4d1c-4519-8772-f8255677733c service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:32:39.813545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10179938-4d1c-4519-8772-f8255677733c service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:32:39.814117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10179938-4d1c-4519-8772-f8255677733c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:32:39.816429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-10179938-4d1c-4519-8772-f8255677733c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:32:39.830710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10179938-4d1c-4519-8772-f8255677733c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:32:39.832768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10179938-4d1c-4519-8772-f8255677733c service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:32:39.846300 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1356/2712] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:32:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:32:47.154681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=32 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:32:49.841801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=40 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:33:25.140132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:33:25.142637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:33:25.221301 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d23e8-d153-4e7d-b0fd-442e7aded1d8 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:33:25.227546 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4318d-fca0-40e5-a73b-cd99d5049513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:33:25.228216 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4318d-fca0-40e5-a73b-cd99d5049513 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:33:25.228506 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4318d-fca0-40e5-a73b-cd99d5049513 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:33:25.228816 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4318d-fca0-40e5-a73b-cd99d5049513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:33:25.236942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4318d-fca0-40e5-a73b-cd99d5049513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:33:25.238518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4318d-fca0-40e5-a73b-cd99d5049513 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:33:25.247553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1357/2713] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:33:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:33:35.243793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=29 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:34:40.151257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10179938-4d1c-4519-8772-f8255677733c service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:40.269491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10179938-4d1c-4519-8772-f8255677733c service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:40.274677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:40.275336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:40.275619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:40.275904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] PATH_INFO: `/v3/users/27eb1354177b48d08fa718ae9bdea996` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:40.277085 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27eb1354177b48d08fa718ae9bdea996)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:40.279020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:40.325248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27eb1354177b48d08fa718ae9bdea996'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.325934 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27eb1354177b48d08fa718ae9bdea996'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.332706 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27eb1354177b48d08fa718ae9bdea996'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.333370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27eb1354177b48d08fa718ae9bdea996'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.342618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27eb1354177b48d08fa718ae9bdea996'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.359722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1357/2714] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:34:40 2022] DELETE /identity/v3/users/27eb1354177b48d08fa718ae9bdea996 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:34:40.364541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:40.365065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:40.365184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:40.384260 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:34:40.385482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:40.450106 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1358/2715] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:34:40.460964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:40.462497 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:40.535172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:34:40.550041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98775c9-8527-4293-8202-6c356c6e5d55 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:40.555314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203da864-6a84-44a0-a961-63db0b17d35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:40.555990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203da864-6a84-44a0-a961-63db0b17d35d service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:40.556269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203da864-6a84-44a0-a961-63db0b17d35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:40.556538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203da864-6a84-44a0-a961-63db0b17d35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:40.566604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203da864-6a84-44a0-a961-63db0b17d35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:40.568096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203da864-6a84-44a0-a961-63db0b17d35d service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:40.577842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1358/2716] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:34:40.863329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:40.865148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:40.878909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03259892-c82f-454c-83d5-86b914e2b7a5 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:40.887796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:40.888480 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:40.888727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:40.889000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] PATH_INFO: `/v3/projects/f9abb24676264ae9824782dfa5d92740` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:40.890170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9abb24676264ae9824782dfa5d92740)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:40.892128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:40.947035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9abb24676264ae9824782dfa5d92740', 'request_id': 'req-97709af6-1019-40e4-9763-6030467a1328'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.953472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9abb24676264ae9824782dfa5d92740', 'request_id': 'req-97709af6-1019-40e4-9763-6030467a1328'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.959666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9abb24676264ae9824782dfa5d92740', 'request_id': 'req-97709af6-1019-40e4-9763-6030467a1328'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.973378 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] The token cache is being invalidate because project f9abb24676264ae9824782dfa5d92740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:34:40.973598 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.974219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-97709af6-1019-40e4-9763-6030467a1328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:34:40.976127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1359/2717] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:34:40 2022] DELETE /identity/v3/projects/f9abb24676264ae9824782dfa5d92740 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:34:40.981900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:40.982187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:40.982483 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.003119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:34:41.004494 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:41.064500 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1359/2718] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:34:41.073022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.073348 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.073625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.094400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:34:41.095645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:41.103732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1360/2719] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:34:41.108126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.118340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b38a9-5a27-4822-93da-dac86bf8891a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.123329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.123995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.124282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.124560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.125265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:34:41.125573 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.127188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.139080 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1360/2720] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:34:41.143150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.152748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476c5d-38fe-4906-9e51-e5b3cb3ef58e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.157964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.158573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.158888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.159138 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.159855 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.161923 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.200540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1361/2721] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:34:41 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:34:41.204998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.218120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad081f0-f1bb-4f27-9cc9-facb8b2d7660 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.280754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.281302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.281587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.281863 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.282544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.284666 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.311835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1361/2722] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:34:41 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:34:41.314651 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.325655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152cbc-a95a-4679-9306-1402bf845654 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.330451 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.331058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.331339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.331613 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.332695 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.334303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.344991 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1362/2723] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:34:41.349057 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.359157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1486e-d6d9-4179-9f2b-d9c86bda0256 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.364119 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8ab04-6fa7-402c-8422-ec64a67db9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.364748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ab04-6fa7-402c-8422-ec64a67db9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.365046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ab04-6fa7-402c-8422-ec64a67db9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.365316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ab04-6fa7-402c-8422-ec64a67db9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.366022 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8ab04-6fa7-402c-8422-ec64a67db9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.367596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8ab04-6fa7-402c-8422-ec64a67db9f2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.380921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1362/2724] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:34:41.385133 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.395811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f4bae-5140-4797-a02e-67d2a7177419 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.400439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.401102 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.401386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.401663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] PATH_INFO: `/v3/projects/291fd3745c9a4b5d9164666b0947dfe4/users/c6f6485eb6674aa597409e120402b2c4/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.416312 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=291fd3745c9a4b5d9164666b0947dfe4, user_id=c6f6485eb6674aa597409e120402b2c4, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.418976 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.434208 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1363/2725] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:34:41 2022] PUT /identity/v3/projects/291fd3745c9a4b5d9164666b0947dfe4/users/c6f6485eb6674aa597409e120402b2c4/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:34:41.502242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.502536 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.502847 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.532968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] MFA Rules not processed for user `c6f6485eb6674aa597409e120402b2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:34:41.534106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:41.574210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:34:41.599183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1363/2726] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:34:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:34:41.611404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:41.613356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:41.637357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:34:41.701826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb28a6-2806-4aa7-bd01-80b572498d02 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:41.705788 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:41.706402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:41.706681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:41.706952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:41.716296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:41.717729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:41.726815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1364/2727] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:34:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:34:51.596693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=9 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:34:51.722798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=26 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:34:51.882094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:34:51.884642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:51.961516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f0eb-a5c0-4d8c-8dbd-62de5b6f534e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:34:51.967756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:34:51.968398 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:34:51.968676 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:34:51.968953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:34:51.970899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:34:52.035267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:34:52.036947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:34:52.046058 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1364/2728] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:34:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:35:02.042083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=56 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:37:35.922156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:37:35.923746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:37:35.999004 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab42ea6-83ed-46b0-b41c-93bc44f0eb5e service glance] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:37:36.004697 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:37:36.005401 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:37:36.005688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:37:36.005980 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:37:36.007885 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:37:36.021277 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:37:36.022886 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:37:36.034126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1365/2729] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:37:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:37:36.575076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:37:36.576648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:37:36.637562 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d871-c3f0-4bb7-ac64-9d751e0256aa service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:37:36.642165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:37:36.643162 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:37:36.643445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:37:36.643715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:37:36.645886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:37:36.660325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:37:36.661768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:37:36.673248 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1365/2730] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:37:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:37:37.193888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:37:37.205665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560374d-254b-4e5f-9206-20fceafbaeea service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:37:37.212148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:37:37.213018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:37:37.213403 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:37:37.213772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:37:37.216541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:37:37.236187 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:37:37.237633 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8874371-a954-4e6d-82a2-24a5ab5c1c5a service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:37:37.248785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1366/2731] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:37:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:37:46.667947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=26 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:37:47.244163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=38 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:38:25.591667 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:38:25.594287 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:25.611326 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6105d05-851d-4aa3-be73-b40f9d0dcaf4 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:38:25.618496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:38:25.619109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:25.619384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:25.619635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:25.621655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:25.708206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:38:25.709812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:38:25.729951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1366/2732] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:38:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:38:35.727266 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=41 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:38:40.212393 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:40.212943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:40.212943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:40.241391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] MFA Rules not processed for user `2637eface78b4813b91967ce66a59c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:38:40.242612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:40.250671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1367/2733] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:38:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:38:40.565178 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:38:40.574529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6648-0dc9-49e1-a839-51a72051ed48 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:38:40.578105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:38:40.578753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:40.579034 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:40.579307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:40.581096 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:40.590572 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:38:40.592020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:38:40.599959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1367/2734] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:38:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:38:40.957268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:38:40.970255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156ed30-e0f9-4189-976f-ab17e0037583 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:38:40.975406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:38:40.976126 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:40.976471 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:40.976814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:40.978670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:40.990880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:38:40.992575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:38:41.004258 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1368/2735] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:38:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:38:43.171174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:38:43.180578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22615a9a-0b86-47da-adf3-bcd527652897 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:38:43.184546 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:38:43.185163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:43.185479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:43.185732 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:43.187583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:43.198029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:38:43.199565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:38:43.208147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1368/2736] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:38:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:38:43.820143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:38:43.834756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0c06-4311-4169-b916-3c2e82466cbd service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:38:43.840837 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:38:43.841457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:43.841733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:43.841999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:43.843992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:43.869583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:38:43.871267 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:38:43.882814 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1369/2737] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:38:47.812198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:38:47.825932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c149e-63d4-4746-a106-397fe78bf72a service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:38:47.831614 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:38:47.832275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:38:47.832558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:38:47.833246 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:38:47.834802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:38:47.847694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:38:47.849307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:38:47.859928 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1369/2738] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:38:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:38:53.878210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=18 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:38:57.855693 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=18 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:40:40.591826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:40:40.593517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:40:40.673770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff78a07-3aa3-4319-b3ad-9f79dc22f750 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:40:40.677944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:40:40.678549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:40:40.678827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:40:40.679096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:40:40.688892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:40:40.690334 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:40:40.718696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1370/2739] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:40:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:40:40.746384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:40:40.756673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c32e8-34a8-4781-9811-a8706452e4cd service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:40:40.761848 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:40:40.762462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:40:40.762742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:40:40.763016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:40:40.772750 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:40:40.774200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:40:40.783130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1370/2740] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:40:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:40:50.715918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=42 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:40:50.779188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:42:41.087118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:42:41.088766 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:42:41.167632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90e750-907d-4a78-9c3a-e1b9899c6ef1 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:42:41.172687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:42:41.173310 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:42:41.173596 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:42:41.173873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:42:41.182469 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:42:41.184130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:42:41.194162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1371/2741] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:42:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:42:51.190081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=26 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:43:25.677459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:43:25.699892 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d494-11d3-4c25-859c-47da03117d5f service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:43:25.703554 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:43:25.704199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:43:25.704482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:43:25.704754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:43:25.714216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:43:25.715658 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:43:25.724768 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1371/2742] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:43:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:43:35.720775 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=9 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:44:38.623944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:44:38.642177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8cbb-b12e-4c22-9eb7-7fc85eaf2cf8 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:44:38.646443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:44:38.647058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:44:38.647365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:44:38.647640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:44:38.655797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:44:38.657286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:44:38.665661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1372/2743] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:44:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:44:48.662069 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=9 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:45:41.803796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:45:41.806553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:45:41.821830 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b8273-f918-4a79-8e25-c5cf67f0bbb0 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:45:41.825898 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:45:41.826509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:45:41.826786 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:45:41.827059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:45:41.828983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:45:41.936747 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:45:41.938177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:45:41.957513 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1372/2744] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:45:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:45:42.004013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:45:42.018163 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b89bc-263e-495f-9e68-cd1b9dc8cd6f service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:45:42.022008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:45:42.022614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:45:42.022907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:45:42.023164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:45:42.025047 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:45:42.096791 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:45:42.098240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:45:42.107237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1373/2745] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:45:51.954677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=49 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:45:52.103344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=35 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:46:40.400591 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:46:40.420285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b5a6e-b436-49bd-84d1-a01aa4c95007 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:46:40.424196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:46:40.424813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:40.425079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:40.425369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:40.500865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:46:40.502302 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:46:40.523047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1373/2746] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:46:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:46:41.714330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:46:41.724611 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ac56-b8d6-434e-be3c-57962a3e0495 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:46:41.728912 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997a510-eacd-48e7-bf43-b77369dd7274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:46:41.729522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997a510-eacd-48e7-bf43-b77369dd7274 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:41.729834 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997a510-eacd-48e7-bf43-b77369dd7274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:41.730137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997a510-eacd-48e7-bf43-b77369dd7274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:41.740876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997a510-eacd-48e7-bf43-b77369dd7274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:46:41.742339 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997a510-eacd-48e7-bf43-b77369dd7274 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:46:41.753576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1374/2747] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:46:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:46:41.852127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:46:41.867641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14857f9-657c-463c-941f-38af80c8fd5a service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:46:41.877983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:46:41.878893 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:41.879271 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:41.879645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:41.893267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:46:41.895734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:46:41.908842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1374/2748] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:46:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:46:44.839499 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:44.840107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:44.840107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:44.859460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] MFA Rules not processed for user `2637eface78b4813b91967ce66a59c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:46:44.860734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:46:44.868582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1375/2749] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:46:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:46:45.458298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:46:45.468089 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2952d2-12d8-4fb7-af01-813e44e844f9 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:46:45.472314 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:46:45.472924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:45.473205 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:45.473475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:45.475337 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:46:45.498088 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:46:45.499516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:46:45.507887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1375/2750] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:46:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:46:45.848906 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:46:45.862087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb8f9-7c71-488d-af34-a9a76329d465 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:46:45.867954 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:46:45.868581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:45.868862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:45.869132 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:45.878292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:46:45.879894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:46:45.890177 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1376/2751] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:46:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:46:54.217365 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:46:54.231622 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c1eaa-06cc-418c-8b43-47342891791b service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:46:54.237552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:46:54.238177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:46:54.238480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:46:54.238755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:46:54.248856 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:46:54.250454 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:46:54.261895 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1376/2752] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:46:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:46:55.886245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=15 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:47:04.257739 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=68 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:47:47.891504 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:47:47.893302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:47:47.965338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836982ac-e702-40ce-9475-7a6abed78d34 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:47:47.971137 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36954a41-641b-4852-9953-db98da862b54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:47:47.971759 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36954a41-641b-4852-9953-db98da862b54 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:47:47.972065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36954a41-641b-4852-9953-db98da862b54 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:47:47.972330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36954a41-641b-4852-9953-db98da862b54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:47:47.974357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-36954a41-641b-4852-9953-db98da862b54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:47:47.985965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36954a41-641b-4852-9953-db98da862b54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:47:47.987544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36954a41-641b-4852-9953-db98da862b54 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:47:47.997977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1377/2753] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:47:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:47:57.993586 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=32 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:48:24.554341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:24.570987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48938b8-2d2e-43a5-b0e5-424a64549561 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:24.578339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1159f64-5172-4a79-a9cc-9a4bc11e6b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:24.578956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159f64-5172-4a79-a9cc-9a4bc11e6b39 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:24.579228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159f64-5172-4a79-a9cc-9a4bc11e6b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:24.579493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159f64-5172-4a79-a9cc-9a4bc11e6b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:24.588980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1159f64-5172-4a79-a9cc-9a4bc11e6b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:24.590406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1159f64-5172-4a79-a9cc-9a4bc11e6b39 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:24.600576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1377/2754] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:48:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:48:34.596376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:48:41.406951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36954a41-641b-4852-9953-db98da862b54 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:41.409184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-36954a41-641b-4852-9953-db98da862b54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:41.510100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36954a41-641b-4852-9953-db98da862b54 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:41.515383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:41.516056 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:41.516336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:41.516624 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] PATH_INFO: `/v3/users/c6f6485eb6674aa597409e120402b2c4` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:41.517983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f6485eb6674aa597409e120402b2c4)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:41.519944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:41.567226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f6485eb6674aa597409e120402b2c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:41.567899 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f6485eb6674aa597409e120402b2c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:41.574942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f6485eb6674aa597409e120402b2c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:41.575646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f6485eb6674aa597409e120402b2c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:41.585256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f6485eb6674aa597409e120402b2c4'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:41.605894 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1378/2755] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:48:41 2022] DELETE /identity/v3/users/c6f6485eb6674aa597409e120402b2c4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:48:41.611127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:41.611845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:41.612922 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:41.641560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:48:41.642793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:41.703476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1378/2756] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:48:41.713340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:41.714831 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:41.774773 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fbfc1-97e5-4d0f-ba8d-79f1be10fd64 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:41.778872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53b46-29d8-4ace-87bc-325a8e5edac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:41.779491 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53b46-29d8-4ace-87bc-325a8e5edac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:41.779801 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53b46-29d8-4ace-87bc-325a8e5edac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:41.780085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53b46-29d8-4ace-87bc-325a8e5edac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:41.786313 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:48:41.790380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f53b46-29d8-4ace-87bc-325a8e5edac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:41.792431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f53b46-29d8-4ace-87bc-325a8e5edac1 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:41.802769 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1379/2757] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:48:42.086840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.088511 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:42.101122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8422c41-7a06-4df3-a1fb-a0877904c8a1 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.106842 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.107514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.108029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.108312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] PATH_INFO: `/v3/projects/291fd3745c9a4b5d9164666b0947dfe4` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.109617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] RBAC: Authorizing `identity:delete_project(project_id=291fd3745c9a4b5d9164666b0947dfe4)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.111753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.162261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291fd3745c9a4b5d9164666b0947dfe4', 'request_id': 'req-95ce377e-3460-4a71-a895-ad9b573dc66e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:42.168028 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291fd3745c9a4b5d9164666b0947dfe4', 'request_id': 'req-95ce377e-3460-4a71-a895-ad9b573dc66e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:42.174268 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291fd3745c9a4b5d9164666b0947dfe4', 'request_id': 'req-95ce377e-3460-4a71-a895-ad9b573dc66e'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:42.188432 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] The token cache is being invalidate because project 291fd3745c9a4b5d9164666b0947dfe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:48:42.188738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:42.189383 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-95ce377e-3460-4a71-a895-ad9b573dc66e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:48:42.190952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1379/2758] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:48:42 2022] DELETE /identity/v3/projects/291fd3745c9a4b5d9164666b0947dfe4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:48:42.196883 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.197180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.197460 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.246713 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:48:42.247989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:42.315031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1380/2759] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:48:42.323111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.323405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.323681 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.344084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:48:42.345267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:42.353687 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1380/2760] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:48:42.358054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.368285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1736e0-fe63-4763-b268-9eb3e8f71cf4 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.373958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.374557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.374862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.375146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.375896 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:48:42.376201 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.377809 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.390030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1381/2761] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:48:42.394575 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.403877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c9b51-f108-41e3-95ec-839eee2ea7a7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.408828 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.409433 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.409714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.409985 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.410745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.412660 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.454192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1381/2762] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:48:42 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:48:42.458898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.469291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d362c-8b69-488c-a11b-53300da9d929 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.529249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.529862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.530147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.530422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.531082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.533035 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.556224 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1382/2763] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:48:42 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:48:42.560324 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.571135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9bdf-7579-4ac6-9ccf-2280be4aabc9 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.576093 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.576708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.577044 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.577352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.578084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.579715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.590102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1382/2764] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:48:42.594131 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.604952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4787bc-fef1-43d4-836a-085969831097 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.611347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a37e36-9a38-45ac-b3fb-e764a52be9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.612180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a37e36-9a38-45ac-b3fb-e764a52be9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.612492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a37e36-9a38-45ac-b3fb-e764a52be9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.612789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a37e36-9a38-45ac-b3fb-e764a52be9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.613496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a37e36-9a38-45ac-b3fb-e764a52be9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.615107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a37e36-9a38-45ac-b3fb-e764a52be9d3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.627085 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1383/2765] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:48:42.631174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.642091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d8ec-fe1e-4709-a979-0bd698bf7483 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.647376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.648068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.648350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.648630 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] PATH_INFO: `/v3/projects/358b2709a9804a1d90825a30e9e334ad/users/bdb8798301404994b0c616696c0c99a2/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.664576 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=358b2709a9804a1d90825a30e9e334ad, user_id=bdb8798301404994b0c616696c0c99a2, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.667334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.681549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1383/2766] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:48:42 2022] PUT /identity/v3/projects/358b2709a9804a1d90825a30e9e334ad/users/bdb8798301404994b0c616696c0c99a2/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:48:42.691745 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.692066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.692340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.722929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] MFA Rules not processed for user `bdb8798301404994b0c616696c0c99a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:48:42.724214 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:42.755646 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:48:42.799881 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1384/2767] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:48:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:48:42.811726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:48:42.814095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:48:42.821710 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:48:42.874874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104870-6cfa-4ca2-8506-32591e0510fa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:48:42.878855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:48:42.879473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:48:42.879754 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:48:42.880074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:48:42.889071 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:48:42.890504 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:48:42.899578 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1384/2768] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:48:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:48:52.797087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=16 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:48:52.895343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=24 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:49:43.595807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:49:43.597494 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:49:43.657746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b819f3-8fdb-421b-8352-cfc7a1821efb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:49:43.661770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:49:43.662383 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:49:43.662663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:49:43.662934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:49:43.664913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:49:43.674709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:49:43.676166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:49:43.684181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1385/2769] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:49:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:49:53.680984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=26 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:50:42.123358 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:50:42.125309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:50:42.161737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5eff76-e59d-4ff8-87e1-b04fbd57cf87 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:50:42.165569 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:50:42.166186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:50:42.166464 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:50:42.166738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:50:42.168535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:50:42.237392 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:50:42.239532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:50:42.267423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1385/2770] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:50:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:50:52.263923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=45 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:35.815833 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:35.817387 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:35.899268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df03787-ebc4-46d2-82ca-8e2abe105842 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:35.903281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244542fc-9f84-421c-bed5-072fd69d363f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a88bb908a0104cc589b19095091a8a53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:35.903930 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244542fc-9f84-421c-bed5-072fd69d363f service glance] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:35.904189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244542fc-9f84-421c-bed5-072fd69d363f service glance] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:35.904486 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244542fc-9f84-421c-bed5-072fd69d363f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:35.906390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-244542fc-9f84-421c-bed5-072fd69d363f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application [None req-244542fc-9f84-421c-bed5-072fd69d363f service glance] Could not find project: 291fd3745c9a4b5d9164666b0947dfe4.: keystone.exception.ProjectNotFound: Could not find project: 291fd3745c9a4b5d9164666b0947dfe4. Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 05 05:51:35.919726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:51:35.921251 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 05:51:35.922757 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application with Lock( Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._enter() Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.creator() Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application created_value = creator( Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 05:51:35.924249 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 05:51:35.925642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 291fd3745c9a4b5d9164666b0947dfe4. Aug 05 05:51:35.925642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: ERROR keystone.server.flask.application Aug 05 05:51:35.925642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1386/2771] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:51:35 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 107 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 05 05:51:44.973106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:45.041960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d23bec-056c-4531-b45c-10a3ac651402 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:45.046986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:45.047619 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.047921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.048196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] PATH_INFO: `/v3/users/bdb8798301404994b0c616696c0c99a2` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.049401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb8798301404994b0c616696c0c99a2)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:45.051377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:45.105406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb8798301404994b0c616696c0c99a2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.106052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb8798301404994b0c616696c0c99a2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.113334 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb8798301404994b0c616696c0c99a2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.113960 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb8798301404994b0c616696c0c99a2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.121621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb8798301404994b0c616696c0c99a2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.139349 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1386/2772] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:51:44 2022] DELETE /identity/v3/users/bdb8798301404994b0c616696c0c99a2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:51:45.144108 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.144638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.144638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.164864 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:45.166097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:45.225309 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1387/2773] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:51:45.239258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:45.240820 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:45.303020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fc866-2d43-4902-9683-98cbdcc25888 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:45.307105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487203c-80e4-4f5a-8e18-7d369610a9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:45.307757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487203c-80e4-4f5a-8e18-7d369610a9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.308074 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487203c-80e4-4f5a-8e18-7d369610a9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.308343 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487203c-80e4-4f5a-8e18-7d369610a9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.319005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487203c-80e4-4f5a-8e18-7d369610a9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:45.320510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487203c-80e4-4f5a-8e18-7d369610a9b3 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:45.330905 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1387/2774] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:51:45.621439 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:45.622880 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:45.650869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879509d7-8dbd-472d-bf1b-2fd06bc8ac9c None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:45.664065 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:45.664888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.665135 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.665432 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] PATH_INFO: `/v3/projects/358b2709a9804a1d90825a30e9e334ad` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.666803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=358b2709a9804a1d90825a30e9e334ad)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:45.669307 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:45.728386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358b2709a9804a1d90825a30e9e334ad', 'request_id': 'req-d89cb97e-1412-4d60-815b-738ad686cb73'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.734872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358b2709a9804a1d90825a30e9e334ad', 'request_id': 'req-d89cb97e-1412-4d60-815b-738ad686cb73'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.746435 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358b2709a9804a1d90825a30e9e334ad', 'request_id': 'req-d89cb97e-1412-4d60-815b-738ad686cb73'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.763290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] The token cache is being invalidate because project 358b2709a9804a1d90825a30e9e334ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:51:45.763860 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.764688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-d89cb97e-1412-4d60-815b-738ad686cb73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:51:45.769760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1388/2775] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:51:45 2022] DELETE /identity/v3/projects/358b2709a9804a1d90825a30e9e334ad => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:51:45.771737 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.772065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.772351 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.794832 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:45.796322 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:45.831657 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:45.916042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1388/2776] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:51:45.921582 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.922000 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.922141 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.942876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:45.944107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:45.952438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1389/2777] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:51:45.957109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:45.967557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ecc94-05f1-4da1-980b-ea48a997c47a None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:45.973006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:45.973774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:45.974355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:45.974355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:45.975015 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:51:45.975380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:45.977038 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:45.989705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1389/2778] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:51:45.993968 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:46.000209 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:46.004392 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccd765-b07d-4dbb-8081-c81f356e6797 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:46.011452 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:46.012115 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.012400 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.012800 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.013477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:46.015576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:46.047220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1390/2779] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:51:45 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:51:46.048777 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:46.059047 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6c7e5-ef50-4b5f-ae1b-bb67f479eb36 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:46.116649 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:46.117294 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.117577 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.117854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.118525 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:46.120563 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:46.143297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1390/2780] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:51:46 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:51:46.148037 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:46.159059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdc06b-8432-4685-810c-d8648c55f012 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:46.163859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:46.164468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.164746 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.165018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.165710 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:46.167440 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:46.177876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1391/2781] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:51:46.181964 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:46.192341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03746c34-b75c-4e0e-a53d-7fcd279abc58 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:46.197170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5192c2-8c34-4de9-a979-51dea357ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:46.197793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5192c2-8c34-4de9-a979-51dea357ef4a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.198076 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5192c2-8c34-4de9-a979-51dea357ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.198355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5192c2-8c34-4de9-a979-51dea357ef4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.199059 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5192c2-8c34-4de9-a979-51dea357ef4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:46.200711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5192c2-8c34-4de9-a979-51dea357ef4a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:46.212558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1391/2782] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:51:46.216827 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:46.226970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526238d5-5dda-4f64-8e82-0b1bc872fb4f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:46.231557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:46.232236 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.232520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.232794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] PATH_INFO: `/v3/projects/e217bd57daf74e9f8e7033fbf4111555/users/9673e5094962449ea96235ca6c397ea3/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.250189 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e217bd57daf74e9f8e7033fbf4111555, user_id=9673e5094962449ea96235ca6c397ea3, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:46.253130 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:46.271149 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1392/2783] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:51:46 2022] PUT /identity/v3/projects/e217bd57daf74e9f8e7033fbf4111555/users/9673e5094962449ea96235ca6c397ea3/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:51:46.276094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009b593-49e2-4ed2-9a41-65212c417549 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.276384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009b593-49e2-4ed2-9a41-65212c417549 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.276654 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009b593-49e2-4ed2-9a41-65212c417549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.296586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-a009b593-49e2-4ed2-9a41-65212c417549 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:46.297821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a009b593-49e2-4ed2-9a41-65212c417549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:46.362148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1392/2784] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:51:46.372030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:46.373887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:46.429887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582995e-ce4a-4232-9ca3-326b05117198 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:46.433558 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:46.434173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:46.434443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:46.434740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:46.443888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:46.445360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:46.455081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1393/2785] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:51:50.449900 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.450436 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.451757 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:50.489046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] MFA Rules not processed for user `9673e5094962449ea96235ca6c397ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:50.491653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:50.513726 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:50.591954 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1393/2786] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:51:50.596597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:50.609210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed664c-c58c-456f-b23e-b358f736dcc9 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:50.624194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:50.625150 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.625434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.625708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:50.626389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:50.628431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:50.665909 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1394/2787] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:51:50 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:51:50.666765 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:50.678515 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962ef9d-1f0a-4146-894a-db009d62fb42 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:50.751168 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:50.752378 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.752904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.753491 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:50.754549 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:50.757113 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:50.781143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1394/2788] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:51:50 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:51:50.785798 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:50.797600 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784f18-b37d-4581-8fea-d7a935ec58d6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:50.802614 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:50.803356 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.803972 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.805537 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:50.806467 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:50.808532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:50.823915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1395/2789] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:51:50.828197 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:50.840024 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa25f06-3768-407d-82de-c6cb0094a452 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:50.844852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:50.845553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.846055 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.846341 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] PATH_INFO: `/v3/projects/68eb7d00193740078c9cb2e03f35b82b/users/2f3284da1b024077a565726bb6a272a0/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:50.865095 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68eb7d00193740078c9cb2e03f35b82b, user_id=2f3284da1b024077a565726bb6a272a0, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:50.867867 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:50.883484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1395/2790] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:51:50 2022] PUT /identity/v3/projects/68eb7d00193740078c9cb2e03f35b82b/users/2f3284da1b024077a565726bb6a272a0/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:51:50.887557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:50.897920 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a6810-9da0-4ae9-a22a-f2629c632de2 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:50.958758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:50.959724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.960022 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.960434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:50.961143 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:50.962843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:50.975281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1396/2791] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:51:50.979759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:50.990699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8681f-95e0-4699-9a5b-86c419ac8df8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:50.996159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe11d0-b53f-4384-8075-f9a9f04a32d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:50.997021 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe11d0-b53f-4384-8075-f9a9f04a32d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:50.997352 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe11d0-b53f-4384-8075-f9a9f04a32d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:50.997697 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe11d0-b53f-4384-8075-f9a9f04a32d0 admin admin] PATH_INFO: `/v3/projects/68eb7d00193740078c9cb2e03f35b82b/users/2f3284da1b024077a565726bb6a272a0/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:51.000389 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe11d0-b53f-4384-8075-f9a9f04a32d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68eb7d00193740078c9cb2e03f35b82b, user_id=2f3284da1b024077a565726bb6a272a0, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:51.003308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe11d0-b53f-4384-8075-f9a9f04a32d0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:51.035261 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1396/2792] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:51:50 2022] PUT /identity/v3/projects/68eb7d00193740078c9cb2e03f35b82b/users/2f3284da1b024077a565726bb6a272a0/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:51:52.576476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:52.578409 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:52.599934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:52.661867 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a7de-ec9c-46a4-948b-33a96046eaef admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:52.740903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:52.741669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:52.742018 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:52.742358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:52.744701 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:52.852741 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:52.854706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:52.871477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1397/2793] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:51:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:51:55.746597 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615507-e391-412c-bafb-9ea3796570d5 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:55.747109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615507-e391-412c-bafb-9ea3796570d5 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:55.747330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55615507-e391-412c-bafb-9ea3796570d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:55.778026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-55615507-e391-412c-bafb-9ea3796570d5 None None] MFA Rules not processed for user `2f3284da1b024077a565726bb6a272a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:55.779282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-55615507-e391-412c-bafb-9ea3796570d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:55.847154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:55.858703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1397/2794] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:51:55.874325 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:55.875919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:55.890979 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e58815-392f-4a46-a7c3-1ee5efd13537 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:55.898268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:55.899086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:55.899521 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:55.900308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:55.982842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:55.984536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:55.988306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:51:55.995482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1398/2795] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:51:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:51:57.280158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:57.281411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:57.282232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:57.319052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] MFA Rules not processed for user `9673e5094962449ea96235ca6c397ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:51:57.321238 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:57.335620 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1398/2796] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:51:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:51:57.813456 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:51:57.832887 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd047-03dc-427a-8739-fd15ad491b10 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:51:57.840505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:51:57.841726 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:51:57.842226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:51:57.842925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:51:57.857821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:51:57.922285 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:51:57.929219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:51:57.962171 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1399/2797] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:51:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:04.742800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:04.803670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15eabf-bb19-4add-8d0c-0b230ff8679d None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:04.808477 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:04.809091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:04.809373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:04.809648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:04.811510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:04.878740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:04.880310 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:04.890335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1399/2798] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:05.379292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:05.382232 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:05.431858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:05.433518 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:05.454685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c4468-71a8-4c8d-9281-dc0495756a90 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:05.459838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:05.460516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:05.460852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:05.461243 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] PATH_INFO: `/v3/users/06de5afa7d2d4d3aa0c1378ceeb5c4bf` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:05.462521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06de5afa7d2d4d3aa0c1378ceeb5c4bf)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:05.464512 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:05.468270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2999ae-61da-45cf-9ede-f4ed77d25657 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:05.473555 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:05.474278 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:05.474623 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:05.474965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:05.477158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:05.490302 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:05.492195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:05.502662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1400/2799] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:05.512646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06de5afa7d2d4d3aa0c1378ceeb5c4bf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.513369 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06de5afa7d2d4d3aa0c1378ceeb5c4bf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.526409 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06de5afa7d2d4d3aa0c1378ceeb5c4bf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.527090 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06de5afa7d2d4d3aa0c1378ceeb5c4bf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.535686 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06de5afa7d2d4d3aa0c1378ceeb5c4bf'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.553282 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1400/2800] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:52:05 2022] DELETE /identity/v3/users/06de5afa7d2d4d3aa0c1378ceeb5c4bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:52:05.742153 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:05.755200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d49a8-3494-402e-b65b-d74c8c7f6bad service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:05.819637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:05.820324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:05.820573 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:05.820844 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] PATH_INFO: `/v3/projects/0af046d32c704d8bb13b821da2b8338a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:05.822016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af046d32c704d8bb13b821da2b8338a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:05.823945 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:05.875758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af046d32c704d8bb13b821da2b8338a', 'request_id': 'req-514346c3-89b6-4b9a-9138-b62e7e9661a0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.881785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af046d32c704d8bb13b821da2b8338a', 'request_id': 'req-514346c3-89b6-4b9a-9138-b62e7e9661a0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.888121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af046d32c704d8bb13b821da2b8338a', 'request_id': 'req-514346c3-89b6-4b9a-9138-b62e7e9661a0'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.902230 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] The token cache is being invalidate because project 0af046d32c704d8bb13b821da2b8338a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:52:05.902516 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.903127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:52:05.904845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1401/2801] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:52:05 2022] DELETE /identity/v3/projects/0af046d32c704d8bb13b821da2b8338a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:52:14.905400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:14.906937 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:14.921431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:14.967731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7225f7-701f-47b1-b850-1d9749f50e53 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:15.017276 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:15.017911 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:15.018190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:15.018461 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:15.020736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:15.093793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:15.095180 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:15.110788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1401/2802] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:15.410145 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:15.411733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:15.418951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:15.483657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514346c3-89b6-4b9a-9138-b62e7e9661a0 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:15.487429 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:15.488079 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:15.488358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:15.488629 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:15.490492 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:15.500981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:15.502433 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:15.510857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1402/2803] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:15.980147 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:15.981642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:16.189280 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999939f2-2af1-42be-82d8-2326f4f3e8c1 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:16.194784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:16.195328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:16.195601 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:16.195907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:16.197855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:16.209610 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:16.211219 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:16.215532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=100 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:16.222668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1402/2804] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:25.507184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=24 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:26.220565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:30.059049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:30.060574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:30.078029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de367531-9b37-4012-a951-d4ab21c7e789 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:30.084152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291097-f340-456e-a05f-43b16b062209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:30.084772 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291097-f340-456e-a05f-43b16b062209 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:30.085053 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291097-f340-456e-a05f-43b16b062209 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:30.085328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291097-f340-456e-a05f-43b16b062209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:30.087298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71291097-f340-456e-a05f-43b16b062209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:30.100399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291097-f340-456e-a05f-43b16b062209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:30.102096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291097-f340-456e-a05f-43b16b062209 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:30.113530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1403/2805] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:32.902883 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:32.914020 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40622b1c-d108-4523-a7c4-ef7adaffc1fd service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:32.918966 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:32.919581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:32.919891 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:32.920159 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:32.922328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:32.987130 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:32.988498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:32.998131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1403/2806] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:52:40.109031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:42.993709 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=30 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:52:52.910297 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291097-f340-456e-a05f-43b16b062209 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:52:52.950026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291097-f340-456e-a05f-43b16b062209 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:52:52.955730 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:52:52.956377 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:52:52.956652 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:52:52.956967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:52:52.959105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:52:52.995419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:52:52.997077 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:52:53.008346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1404/2807] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:52:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:53:03.003360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=25 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:53:24.704181 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:53:24.706413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:53:24.735718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f3890-341c-4a72-8e10-81522c6237be service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:53:24.740602 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:53:24.741217 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:53:24.741490 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:53:24.741756 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:53:24.743865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:53:24.826662 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:53:24.828301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:53:24.847346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1404/2808] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:53:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:53:34.844729 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=45 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:54:42.475134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:54:42.496852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3258b-c7d9-4b46-a231-c259f2703a62 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:54:42.501263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:54:42.501826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:54:42.502107 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:54:42.502379 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:54:42.504729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:54:42.514472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:54:42.515955 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:54:42.524006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1405/2809] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:54:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:54:43.669142 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:54:43.670677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:54:43.691126 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42d52d-75ed-4288-9f04-79e80b90d6c2 service cinder] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:54:43.695384 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:54:43.696053 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:54:43.696330 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:54:43.696598 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:54:43.698417 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:54:43.708431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:54:43.709860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:54:43.717707 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1405/2810] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:54:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:54:43.816929 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:54:43.825657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf962-2dd9-4191-8c64-bd9a5127d055 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:54:43.829269 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:54:43.829869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:54:43.830148 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:54:43.830417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:54:43.832121 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:54:43.844892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:54:43.846326 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:54:43.856997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1406/2811] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:54:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:54:45.398267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:54:45.407810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a956-5479-4b5e-9edd-cf81f8b5af00 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:54:45.412882 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:54:45.413516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:54:45.413788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:54:45.414056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:54:45.416193 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:54:45.426353 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:54:45.427979 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:54:45.436312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1406/2812] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:54:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:54:53.852597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=15 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:54:55.432798 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=15 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:16.200041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:16.245041 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7ef2a-6458-4f80-baea-95eee31f7b20 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:16.252708 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:16.253399 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:16.253775 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:16.254176 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:16.256386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:16.269428 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:16.271164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:16.281771 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1407/2813] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:56:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:56:19.183812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:19.186004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:19.254394 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66073d-64ec-48ab-bbc3-20935ccbd461 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:19.259731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:19.260404 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:19.260683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:19.260956 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] PATH_INFO: `/v3/users/9673e5094962449ea96235ca6c397ea3` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:19.262167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9673e5094962449ea96235ca6c397ea3)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:19.264158 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:19.314834 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9673e5094962449ea96235ca6c397ea3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.315618 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9673e5094962449ea96235ca6c397ea3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.323395 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9673e5094962449ea96235ca6c397ea3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.324065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9673e5094962449ea96235ca6c397ea3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.333045 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9673e5094962449ea96235ca6c397ea3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.350187 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1407/2814] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:56:19 2022] DELETE /identity/v3/users/9673e5094962449ea96235ca6c397ea3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:19.538989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:19.550274 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04cfe6-6ed2-41c9-9e10-970aa92384b7 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:19.611225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:19.611904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:19.612191 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:19.612472 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] PATH_INFO: `/v3/projects/e217bd57daf74e9f8e7033fbf4111555` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:19.613645 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e217bd57daf74e9f8e7033fbf4111555)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:19.615580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:19.665372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e217bd57daf74e9f8e7033fbf4111555', 'request_id': 'req-e4cc6032-9573-4b08-ade2-f38317ca6ff6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.671147 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e217bd57daf74e9f8e7033fbf4111555', 'request_id': 'req-e4cc6032-9573-4b08-ade2-f38317ca6ff6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.677684 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e217bd57daf74e9f8e7033fbf4111555', 'request_id': 'req-e4cc6032-9573-4b08-ade2-f38317ca6ff6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.691335 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] The token cache is being invalidate because project e217bd57daf74e9f8e7033fbf4111555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:56:19.691630 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.692376 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.694162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1408/2815] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:56:19 2022] DELETE /identity/v3/projects/e217bd57daf74e9f8e7033fbf4111555 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:19.698551 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:19.700072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:19.712967 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b457ea1-2720-491a-afd7-58b175a7fa9b admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:19.769405 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:19.770031 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:19.770308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:19.770579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] PATH_INFO: `/v3/users/2f3284da1b024077a565726bb6a272a0` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:19.771800 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3284da1b024077a565726bb6a272a0)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:19.773692 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:19.789254 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:19.821741 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3284da1b024077a565726bb6a272a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.822380 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3284da1b024077a565726bb6a272a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.833441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3284da1b024077a565726bb6a272a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.834137 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3284da1b024077a565726bb6a272a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.844340 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-63da6f84-0d8f-4314-9050-ef4b558cc9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3284da1b024077a565726bb6a272a0'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:19.862740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1408/2816] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:56:19 2022] DELETE /identity/v3/users/2f3284da1b024077a565726bb6a272a0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:20.047477 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.049122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:20.063442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc6032-9573-4b08-ade2-f38317ca6ff6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.122952 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.123602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.123918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.124188 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] PATH_INFO: `/v3/projects/68eb7d00193740078c9cb2e03f35b82b` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.125414 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] RBAC: Authorizing `identity:delete_project(project_id=68eb7d00193740078c9cb2e03f35b82b)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.127748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.129350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:20.178268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68eb7d00193740078c9cb2e03f35b82b', 'request_id': 'req-92573533-5eeb-48a8-ae1c-c224ef4a148d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:20.184789 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68eb7d00193740078c9cb2e03f35b82b', 'request_id': 'req-92573533-5eeb-48a8-ae1c-c224ef4a148d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:20.193423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68eb7d00193740078c9cb2e03f35b82b', 'request_id': 'req-92573533-5eeb-48a8-ae1c-c224ef4a148d'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:20.207241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] The token cache is being invalidate because project 68eb7d00193740078c9cb2e03f35b82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:56:20.207540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:20.208156 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-92573533-5eeb-48a8-ae1c-c224ef4a148d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:56:20.209671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1409/2817] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:56:20 2022] DELETE /identity/v3/projects/68eb7d00193740078c9cb2e03f35b82b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:20.214823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.215281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.215403 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.236459 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:56:20.237822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:20.301226 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1409/2818] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:56:20.310217 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.310514 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.310793 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.332013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:56:20.333256 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:20.342532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1410/2819] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:56:20.347041 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.357216 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065b394-08af-4d0b-adde-1f8515193ca7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.363070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.363685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.363991 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.364242 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.364940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:56:20.365236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.366845 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.379179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1410/2820] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:56:20.383533 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.393255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99543da-d317-453f-8e4c-c4ba1c754a36 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.399391 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.400045 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.400337 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.400606 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.401272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.403204 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.435021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1411/2821] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:56:20 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:56:20.440669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.451029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58833cb-a24a-418d-81c0-48d224335b6a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.509097 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.509721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.510007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.510285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.510955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.512907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.535983 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1411/2822] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:56:20 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:56:20.540271 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.550705 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d9b7c-89dd-4f58-a02f-be201574659e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.556281 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.556888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.557167 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.557436 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.558164 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.559796 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.570263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1412/2823] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:56:20.574482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.585319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d36d8-c82e-4ca2-a654-9666b9592aa8 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.591422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8c1f6-0699-42b7-9f98-c27ae62c87bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.592070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8c1f6-0699-42b7-9f98-c27ae62c87bb admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.592346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8c1f6-0699-42b7-9f98-c27ae62c87bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.592626 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8c1f6-0699-42b7-9f98-c27ae62c87bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.593354 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8c1f6-0699-42b7-9f98-c27ae62c87bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.594962 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8c1f6-0699-42b7-9f98-c27ae62c87bb admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.600749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:20.606596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1412/2824] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:56:20.611146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.621583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd9b9-01f0-41cb-9991-8cb4ccd0d59c admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.626657 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.627670 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.628015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.628303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] PATH_INFO: `/v3/projects/d93689b1aaa347368f1f590eb6b494cb/users/4f727e9d632e4592b82ebd1f00cdf8df/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.643370 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93689b1aaa347368f1f590eb6b494cb, user_id=4f727e9d632e4592b82ebd1f00cdf8df, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.646080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.659475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1413/2825] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:56:20 2022] PUT /identity/v3/projects/d93689b1aaa347368f1f590eb6b494cb/users/4f727e9d632e4592b82ebd1f00cdf8df/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:20.664653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e21b-b65b-474c-a29f-b33a5b6ff376 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.664973 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e21b-b65b-474c-a29f-b33a5b6ff376 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.665247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e21b-b65b-474c-a29f-b33a5b6ff376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.684670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-6585e21b-b65b-474c-a29f-b33a5b6ff376 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:56:20.685952 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-6585e21b-b65b-474c-a29f-b33a5b6ff376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:20.748114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1413/2826] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:56:20.764350 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:20.767953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:20.839545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624ba1b-f5e5-4d18-9b85-ac06b460878f admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:20.843936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:20.844557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:20.844845 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:20.845117 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:20.857828 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:20.859283 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:20.869505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1414/2827] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:56:24.854441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:24.854974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:24.854974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:24.887495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] MFA Rules not processed for user `4f727e9d632e4592b82ebd1f00cdf8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:56:24.888690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:24.959123 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1414/2828] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:56:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:56:24.963241 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:24.975016 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf639c7-fa3d-43a1-be76-7ac71a3aaea5 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:24.979977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:24.980680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:24.981095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:24.981489 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:24.982336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:24.984366 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:25.039005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1415/2829] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:56:24 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:56:25.040154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:25.051116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3997-7f21-4a7a-9461-914a53eebd60 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:25.114852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:25.115496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:25.115788 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:25.116068 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:25.116738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:25.118685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:25.144564 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1415/2830] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:56:25 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:56:25.148754 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:25.159662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9031dd2-ef14-4120-8797-8ecb2abe0376 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:25.164422 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:25.165049 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:25.165328 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:25.165602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:25.166381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:25.167966 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:25.169454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:25.180785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1416/2831] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:56:25.185152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:25.195742 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dc9a4-d1d7-4125-b4a7-2139ed111e46 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:25.200745 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:25.201442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:25.201723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:25.201999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] PATH_INFO: `/v3/projects/3b1cb7bfce7c429e907c367ea58ff694/users/17c523f9acaa4444b706ef51559031c6/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:25.217668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1cb7bfce7c429e907c367ea58ff694, user_id=17c523f9acaa4444b706ef51559031c6, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:25.220429 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:25.223727 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:25.237121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1416/2832] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:56:25 2022] PUT /identity/v3/projects/3b1cb7bfce7c429e907c367ea58ff694/users/17c523f9acaa4444b706ef51559031c6/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:25.241280 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:25.251853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45746193-1e66-48b4-8aeb-ad141770ba61 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:25.312360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8418cff-84e0-4cc1-a237-fcc644888af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:25.312994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8418cff-84e0-4cc1-a237-fcc644888af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:25.313270 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8418cff-84e0-4cc1-a237-fcc644888af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:25.313728 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8418cff-84e0-4cc1-a237-fcc644888af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:25.314434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8418cff-84e0-4cc1-a237-fcc644888af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:25.316109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8418cff-84e0-4cc1-a237-fcc644888af0 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:25.326737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1417/2833] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:56:25.331183 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:25.342721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137a9a1-2fdf-452c-8ccd-b2ab6a8596aa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:25.347840 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:25.348600 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:25.348965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:25.349307 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] PATH_INFO: `/v3/projects/3b1cb7bfce7c429e907c367ea58ff694/users/17c523f9acaa4444b706ef51559031c6/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:25.351375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1cb7bfce7c429e907c367ea58ff694, user_id=17c523f9acaa4444b706ef51559031c6, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:25.354154 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:25.366624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1417/2834] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:56:25 2022] PUT /identity/v3/projects/3b1cb7bfce7c429e907c367ea58ff694/users/17c523f9acaa4444b706ef51559031c6/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:56:29.635225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7177b-1ed8-47b7-a25f-f57ef814ac6b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:29.635852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7177b-1ed8-47b7-a25f-f57ef814ac6b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:29.635964 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7177b-1ed8-47b7-a25f-f57ef814ac6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:29.671096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-8ad7177b-1ed8-47b7-a25f-f57ef814ac6b None None] MFA Rules not processed for user `17c523f9acaa4444b706ef51559031c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:56:29.672448 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-8ad7177b-1ed8-47b7-a25f-f57ef814ac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:29.750729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1418/2835] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:56:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:56:29.760865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:29.762921 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:29.841027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8b56f-ae3c-458b-9650-738c1c5a4365 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:29.847213 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:29.847880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:29.848192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:29.848467 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:29.910791 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:29.912818 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:29.922345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1418/2836] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:56:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:56:39.740642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=65 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:39.917853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:56:46.337760 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:46.338226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:46.338316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:46.358673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] MFA Rules not processed for user `4f727e9d632e4592b82ebd1f00cdf8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:56:46.359898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:46.367892 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1419/2837] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:56:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:56:53.083152 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:56:53.161978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41904677-b9f7-43c8-952e-ec8f261dd875 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:56:53.165948 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:56:53.166565 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:56:53.166837 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:56:53.167105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:56:53.169201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:56:53.189249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:56:53.190664 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:56:53.212109 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1419/2838] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:56:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:56:56.352857 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:57:01.832505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:57:01.847680 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843585c-fa45-4362-9869-b08c4992f1fb None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:57:01.854246 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:57:01.855109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:57:01.855505 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:57:01.855922 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:57:01.867634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:57:01.869119 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:57:01.879090 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1420/2839] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:57:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:57:03.209186 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=37 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:57:03.788274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:57:03.789724 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:57:03.803125 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb69e7-dfd0-41cc-8add-b688dc2bb853 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:57:03.809456 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:57:03.810164 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:57:03.810519 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:57:03.810858 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:57:03.812917 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:57:03.826110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:57:03.827685 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:57:03.838678 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1420/2840] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:57:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:57:11.874758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:57:13.834829 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=6 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:57:56.839185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:57:56.840865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:57:56.915574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb3642-6ad3-4961-88d3-1d1451f84ad5 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:57:56.921092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:57:56.921706 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:57:56.921989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:57:56.922255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:57:56.924192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:57:56.936331 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:57:56.937919 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:57:56.948033 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1421/2841] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:57:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:02.198784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:02.210699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa799-e654-4458-87c0-f1a78c243a30 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:02.214634 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:02.215253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:02.215530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:02.215838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:02.217663 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:02.283810 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:02.285195 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:02.294873 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1421/2842] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:05.567184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:05.569344 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:05.589134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755eca3-30a4-4c22-9ab2-26de965e188e service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:05.594286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:05.594925 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:05.595233 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:05.595509 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] PATH_INFO: `/v3/users/2637eface78b4813b91967ce66a59c00` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:05.596758 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2637eface78b4813b91967ce66a59c00)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:05.598679 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:05.659672 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2637eface78b4813b91967ce66a59c00'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:05.660364 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2637eface78b4813b91967ce66a59c00'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:05.667365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2637eface78b4813b91967ce66a59c00'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:05.668030 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2637eface78b4813b91967ce66a59c00'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:05.676093 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-341e993b-3a25-45ba-83d8-102d3a7afd74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2637eface78b4813b91967ce66a59c00'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:05.694417 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1422/2843] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:05 2022] DELETE /identity/v3/users/2637eface78b4813b91967ce66a59c00 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:58:05.882779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:05.893716 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc457c2-1b8f-4aad-9aec-47f3c2b697d4 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:05.953342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:05.954121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:05.954401 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:05.954680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] PATH_INFO: `/v3/projects/e45c617eb5804c8187985b40e94329d7` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:05.955936 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45c617eb5804c8187985b40e94329d7)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:05.957826 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.014533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45c617eb5804c8187985b40e94329d7', 'request_id': 'req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:06.020172 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45c617eb5804c8187985b40e94329d7', 'request_id': 'req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:06.026412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45c617eb5804c8187985b40e94329d7', 'request_id': 'req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:06.041005 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] The token cache is being invalidate because project e45c617eb5804c8187985b40e94329d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:06.041292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:06.041944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-9ad35dc2-d61a-413d-b02f-e85fb2e2cca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:06.043621 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1422/2844] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:05 2022] DELETE /identity/v3/projects/e45c617eb5804c8187985b40e94329d7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:06.049853 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.050146 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.050424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.070615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:06.071869 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:06.138183 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1423/2845] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:06.147190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.147478 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.147749 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.168669 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:06.169874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:06.178869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1423/2846] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:06.185612 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.193308 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:06.203275 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cf00e-a61f-402d-a7eb-f236f5ae3d36 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.211527 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.212292 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.212581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.212862 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.213566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:06.213875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.215496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.231635 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1424/2847] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:06.232668 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.242435 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa294a-f280-49a2-b4a3-7cf6723eba17 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.247860 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.248479 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.248758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.249027 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.249731 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.252067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.268744 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:06.284052 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1424/2848] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:06 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:06.288566 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.300413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631368bf-e3b4-4286-a7ad-0c0e7583209e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.357320 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.357944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.358223 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.358495 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.359200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.361162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.384276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1425/2849] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:06 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:06.388499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.400705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d34aaf-8e6b-4bc1-9b0b-06d4113d2541 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.407252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.407995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.408281 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.408550 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.409251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.410857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.421442 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1425/2850] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:06.425627 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.436082 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c064bb-002c-4508-b5b4-ee290901151a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.441685 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adeb9d-a0bc-43af-9623-bd388f754cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.442290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adeb9d-a0bc-43af-9623-bd388f754cab admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.442568 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adeb9d-a0bc-43af-9623-bd388f754cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.442876 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adeb9d-a0bc-43af-9623-bd388f754cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.443615 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adeb9d-a0bc-43af-9623-bd388f754cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.445371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adeb9d-a0bc-43af-9623-bd388f754cab admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.456443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1426/2851] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:06.460452 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.470492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7b62-41f2-407b-a1c6-b64e4b741caa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.475990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.476708 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.476995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.477258 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] PATH_INFO: `/v3/projects/c46538f0eda54a198f3085ca9a4d097f/users/c7c75eebcb6846309aaf2e83215f5552/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.492509 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46538f0eda54a198f3085ca9a4d097f, user_id=c7c75eebcb6846309aaf2e83215f5552, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.495151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.510648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1426/2852] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:06 2022] PUT /identity/v3/projects/c46538f0eda54a198f3085ca9a4d097f/users/c7c75eebcb6846309aaf2e83215f5552/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:58:06.523476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.523792 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.524118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.558643 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] MFA Rules not processed for user `c7c75eebcb6846309aaf2e83215f5552`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:06.559818 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:06.626424 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1427/2853] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:06.637888 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.639942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:06.715779 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b2e7-a3bd-4066-ac61-de04e6ac3127 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.722082 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.722696 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.722971 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.723251 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.731325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.732933 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.742046 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1427/2854] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:06.821075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:06.822882 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:06.883743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d8d98-5797-41b2-8045-701237bb5f88 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:06.887926 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:06.888541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:06.888826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:06.889100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:06.892013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:06.909142 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:06.910584 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481ae28-962c-4d82-821c-6a0bbb1f364d service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:06.916340 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:06.921663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1428/2855] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:07.060101 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:07.124070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfaf314-0f2a-4727-8b30-c6c309dafbb2 service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:07.129377 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:07.130008 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.130288 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.130538 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] PATH_INFO: `/v3/users/c7c75eebcb6846309aaf2e83215f5552` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.131787 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c75eebcb6846309aaf2e83215f5552)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:07.133774 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:07.184545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c75eebcb6846309aaf2e83215f5552'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.185192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c75eebcb6846309aaf2e83215f5552'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.186368 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:07.193942 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c75eebcb6846309aaf2e83215f5552'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.194864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c75eebcb6846309aaf2e83215f5552'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.203185 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c75eebcb6846309aaf2e83215f5552'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.221156 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1428/2856] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:07 2022] DELETE /identity/v3/users/c7c75eebcb6846309aaf2e83215f5552 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:07.225825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.226109 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.226450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.246849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:07.248134 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:07.308849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1429/2857] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:07.319023 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:07.320801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:07.389839 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1bb2c-9cb0-4cd4-8292-57442b7cd57c admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:07.393628 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698257-6ee6-4c21-a46a-cc6319db0a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:07.394240 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698257-6ee6-4c21-a46a-cc6319db0a15 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.394528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698257-6ee6-4c21-a46a-cc6319db0a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.394802 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698257-6ee6-4c21-a46a-cc6319db0a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.403974 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698257-6ee6-4c21-a46a-cc6319db0a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:07.405411 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698257-6ee6-4c21-a46a-cc6319db0a15 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:07.414795 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1429/2858] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:07.692682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:07.694664 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:07.709626 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cf4db-94f0-4352-8f0b-2eb1d76142ee None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:07.717152 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:07.717868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.718172 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.719421 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] PATH_INFO: `/v3/projects/c46538f0eda54a198f3085ca9a4d097f` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.721225 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46538f0eda54a198f3085ca9a4d097f)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:07.723756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:07.774272 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46538f0eda54a198f3085ca9a4d097f', 'request_id': 'req-4460e53f-6103-4d96-886a-b0f4a86edeb5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.779957 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46538f0eda54a198f3085ca9a4d097f', 'request_id': 'req-4460e53f-6103-4d96-886a-b0f4a86edeb5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.786103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46538f0eda54a198f3085ca9a4d097f', 'request_id': 'req-4460e53f-6103-4d96-886a-b0f4a86edeb5'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.799654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] The token cache is being invalidate because project c46538f0eda54a198f3085ca9a4d097f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:07.799970 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.800575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-4460e53f-6103-4d96-886a-b0f4a86edeb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:07.802058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1430/2859] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:07 2022] DELETE /identity/v3/projects/c46538f0eda54a198f3085ca9a4d097f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:07.807309 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.807604 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.807935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.828495 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:07.829720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:07.891480 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1430/2860] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:07.899737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.900052 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.900333 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.920029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:07.921220 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:07.929336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1431/2861] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:07.933469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:07.944177 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9094-719c-48ea-b449-8c74486cd9e2 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:07.949704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:07.950311 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.950587 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.950854 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.951544 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:07.951896 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:07.953496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:07.965176 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1431/2862] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:07.969295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:07.979011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079db44-5d83-4e55-a44c-1f6518bb42a3 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:07.984647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:07.985273 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:07.985543 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:07.985817 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:07.986474 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:07.988450 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.018031 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1432/2863] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:07 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:08.022265 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.032220 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a7da9-721c-4016-8bd2-e4634434e4ef admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.072776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:08.089335 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.089949 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.090225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.090496 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.091141 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.093100 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.115532 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1432/2864] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:08 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:58:08.119752 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.130852 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff929306-6407-4e24-b571-75ed2e34514d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.135942 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.136667 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.136927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.137363 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.138266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.140173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.150159 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1433/2865] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:08.154122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.165091 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a0a42-8ddb-4794-8410-285df99cb6ac admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.171155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6b532-222b-430a-9af1-1b18539d75d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.171801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6b532-222b-430a-9af1-1b18539d75d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.172116 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6b532-222b-430a-9af1-1b18539d75d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.172372 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6b532-222b-430a-9af1-1b18539d75d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.173067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6b532-222b-430a-9af1-1b18539d75d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.174653 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6b532-222b-430a-9af1-1b18539d75d3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.185557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1433/2866] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:08.189767 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.200242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e0a41-8229-4f9c-8dc6-8dc287a9bd5e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.205342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.205987 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.206263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.206536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] PATH_INFO: `/v3/projects/860039b2f8a745d0be3007b71a6ccf20/users/f62b5da835934d1387a75e49f567c44a/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.222454 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=860039b2f8a745d0be3007b71a6ccf20, user_id=f62b5da835934d1387a75e49f567c44a, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.225151 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.240219 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1434/2867] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:08 2022] PUT /identity/v3/projects/860039b2f8a745d0be3007b71a6ccf20/users/f62b5da835934d1387a75e49f567c44a/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:08.252718 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.252978 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.253252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.280806 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] MFA Rules not processed for user `f62b5da835934d1387a75e49f567c44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:08.281969 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:08.348382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1434/2868] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:08.353054 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.374918 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:08.418636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa461070-cd54-461e-ac1a-fbb82de5bd5e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.423755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.424404 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.424681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.424951 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.425625 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.427536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.461075 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1435/2869] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:08 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:58:08.465362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.476624 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a5b1-f422-4ad3-b8d6-081dc02bc758 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.536605 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.537206 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.537499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.537785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.538439 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.540514 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.563908 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1435/2870] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:08 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:08.567982 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.578438 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1acc90-9391-4cd2-8d9c-0f0ca6f86032 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.583029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.583641 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.583961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.584242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.584943 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.587104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.597879 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1436/2871] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:08.602070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.612236 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47c6ba-3f36-429a-b14b-928e4f16daba admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.617474 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.618127 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.618407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.618683 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] PATH_INFO: `/v3/projects/c6c738d3ed3c41ec8a2ed4fc18287f92/users/47473e4cb9aa45019fed2c03f440e502/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.634016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c738d3ed3c41ec8a2ed4fc18287f92, user_id=47473e4cb9aa45019fed2c03f440e502, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.636670 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.651884 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1436/2872] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:08 2022] PUT /identity/v3/projects/c6c738d3ed3c41ec8a2ed4fc18287f92/users/47473e4cb9aa45019fed2c03f440e502/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:08.655985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.666406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafdea4-ca43-40bc-8f0f-8aa5c7088ad6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.724859 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318aab8-2217-43d7-8aa0-89daf1f5770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.725470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318aab8-2217-43d7-8aa0-89daf1f5770f admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.725677 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318aab8-2217-43d7-8aa0-89daf1f5770f admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.725958 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318aab8-2217-43d7-8aa0-89daf1f5770f admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.726734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318aab8-2217-43d7-8aa0-89daf1f5770f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.728390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318aab8-2217-43d7-8aa0-89daf1f5770f admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.738468 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1437/2873] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:08.742951 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.754135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa02fca-beaf-49e0-b7a9-35d77f00aed6 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.759725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.760418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.760703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.760961 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] PATH_INFO: `/v3/projects/c6c738d3ed3c41ec8a2ed4fc18287f92/users/47473e4cb9aa45019fed2c03f440e502/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.763006 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c738d3ed3c41ec8a2ed4fc18287f92, user_id=47473e4cb9aa45019fed2c03f440e502, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:08.765645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:08.775838 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1437/2874] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:08 2022] PUT /identity/v3/projects/c6c738d3ed3c41ec8a2ed4fc18287f92/users/47473e4cb9aa45019fed2c03f440e502/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:08.785990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.786286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.786567 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.815180 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] MFA Rules not processed for user `47473e4cb9aa45019fed2c03f440e502`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:08.816420 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:08.899714 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1438/2875] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:58:08.911508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:08.913407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:08.920342 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:08.978965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1318e-af7c-4654-ad96-1125342d68e0 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:08.984111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c5f76-7e77-42e1-ad3d-caab0a4d7a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:08.985002 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c5f76-7e77-42e1-ad3d-caab0a4d7a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:08.985357 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c5f76-7e77-42e1-ad3d-caab0a4d7a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:08.985702 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c5f76-7e77-42e1-ad3d-caab0a4d7a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:08.998345 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c5f76-7e77-42e1-ad3d-caab0a4d7a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:09.000198 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c5f76-7e77-42e1-ad3d-caab0a4d7a98 service cinder] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:09.010318 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1438/2876] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:09.431648 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:09.453863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:09.501632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71688124-b117-457d-93f1-fca9a2ee7441 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:09.506785 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:09.507549 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:09.507890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:09.508162 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] PATH_INFO: `/v3/users/f62b5da835934d1387a75e49f567c44a` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:09.509386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62b5da835934d1387a75e49f567c44a)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:09.511371 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:09.560938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62b5da835934d1387a75e49f567c44a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:09.561578 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62b5da835934d1387a75e49f567c44a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:09.569244 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62b5da835934d1387a75e49f567c44a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:09.569973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62b5da835934d1387a75e49f567c44a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:09.577986 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62b5da835934d1387a75e49f567c44a'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:09.596682 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1439/2877] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:09 2022] DELETE /identity/v3/users/f62b5da835934d1387a75e49f567c44a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:58:09.601636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:09.602111 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:09.602223 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:09.622418 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:09.623645 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:09.685121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1439/2878] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:09.695294 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:09.696733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:09.758386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831e9dc-a671-4d0e-9da9-daec2678b2fa admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:09.762413 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:09.763027 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:09.763330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:09.763601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:09.773381 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:09.774811 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:09.786807 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1440/2879] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:10.324292 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:10.325796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:10.340214 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dfaf3-74e6-4dab-b9eb-05896845eca3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:10.345941 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:10.346594 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:10.346869 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:10.347157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] PATH_INFO: `/v3/projects/860039b2f8a745d0be3007b71a6ccf20` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:10.348453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=860039b2f8a745d0be3007b71a6ccf20)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:10.350424 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:10.398992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860039b2f8a745d0be3007b71a6ccf20', 'request_id': 'req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.404599 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860039b2f8a745d0be3007b71a6ccf20', 'request_id': 'req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.410887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860039b2f8a745d0be3007b71a6ccf20', 'request_id': 'req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.427633 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] The token cache is being invalidate because project 860039b2f8a745d0be3007b71a6ccf20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:10.427955 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.428770 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.430510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1440/2880] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:10 2022] DELETE /identity/v3/projects/860039b2f8a745d0be3007b71a6ccf20 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:10.434628 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:10.444890 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a59c1-b5d9-47b0-a4a7-1d74b1446be9 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:10.481500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:10.506006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:10.506642 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:10.506921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:10.507196 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] PATH_INFO: `/v3/users/47473e4cb9aa45019fed2c03f440e502` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:10.508427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=47473e4cb9aa45019fed2c03f440e502)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:10.510355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:10.563259 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47473e4cb9aa45019fed2c03f440e502'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.563928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47473e4cb9aa45019fed2c03f440e502'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.571315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47473e4cb9aa45019fed2c03f440e502'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.571977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47473e4cb9aa45019fed2c03f440e502'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.580815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3f6508c5-f4f8-4357-bca6-618d0b890ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47473e4cb9aa45019fed2c03f440e502'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:10.598266 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1441/2881] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:10 2022] DELETE /identity/v3/users/47473e4cb9aa45019fed2c03f440e502 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:10.890422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:10.891947 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:10.905879 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbeed5b-0201-44e0-b6bc-b4b10660aba4 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:10.938148 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:10.968175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:10.968794 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:10.969075 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:10.969350 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] PATH_INFO: `/v3/projects/c6c738d3ed3c41ec8a2ed4fc18287f92` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:10.970557 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c738d3ed3c41ec8a2ed4fc18287f92)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:10.972510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.020560 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c738d3ed3c41ec8a2ed4fc18287f92', 'request_id': 'req-b4f06d06-a600-4dec-989f-ead571401715'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:11.026175 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c738d3ed3c41ec8a2ed4fc18287f92', 'request_id': 'req-b4f06d06-a600-4dec-989f-ead571401715'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:11.032328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c738d3ed3c41ec8a2ed4fc18287f92', 'request_id': 'req-b4f06d06-a600-4dec-989f-ead571401715'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:11.050312 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] The token cache is being invalidate because project c6c738d3ed3c41ec8a2ed4fc18287f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:11.050586 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:11.051225 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-b4f06d06-a600-4dec-989f-ead571401715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:11.052793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1441/2882] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:10 2022] DELETE /identity/v3/projects/c6c738d3ed3c41ec8a2ed4fc18287f92 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:11.061402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.061916 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.062073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.088296 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:11.089544 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:11.167304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1442/2883] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:11.175413 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.175690 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.175996 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.195642 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:11.196857 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:11.204723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1442/2884] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:58:11.208961 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.219095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c6530-7a1b-4a6a-8297-a39008093f8e None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.223932 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.224541 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.224824 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.225096 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.225790 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80378) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:11.226095 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.227707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.240365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1443/2885] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:11.244841 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.254652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362784f-ef28-4ca9-8c76-9ce0a439924c None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.260408 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.261004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.261263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.261534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.262199 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.264174 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.291722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1443/2886] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:58:11.296154 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.306242 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e62295-135e-4014-922e-44f506de9a9d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.354021 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:11.376213 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.376842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.377122 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.377396 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.378080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.380055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.400597 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1444/2887] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:58:11.405865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.416714 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd4914-0120-4456-97c3-fb15a463ff5e admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.422200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.422819 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.423099 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.423370 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.424110 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.426740 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.441445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1444/2888] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:11.445905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.456875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f8a35-de7f-48b1-8784-558717cee175 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.462770 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7df80-468b-40af-991a-8dad48155db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.463418 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7df80-468b-40af-991a-8dad48155db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.463699 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7df80-468b-40af-991a-8dad48155db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.464006 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7df80-468b-40af-991a-8dad48155db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.464702 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7df80-468b-40af-991a-8dad48155db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.466329 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7df80-468b-40af-991a-8dad48155db4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.477985 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1445/2889] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:11.482407 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.493094 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25346ee4-3329-4256-8c7f-9105cc7f028a admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.498835 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.499506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.499913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.500067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] PATH_INFO: `/v3/projects/463fbe6479e14c799b1a2186bc5e3466/users/2690bdc0630f43e5b05cbcd9281303d5/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.515255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=463fbe6479e14c799b1a2186bc5e3466, user_id=2690bdc0630f43e5b05cbcd9281303d5, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.517880 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.533652 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1445/2890] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:11 2022] PUT /identity/v3/projects/463fbe6479e14c799b1a2186bc5e3466/users/2690bdc0630f43e5b05cbcd9281303d5/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:11.545235 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.545637 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.545913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.576688 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] MFA Rules not processed for user `2690bdc0630f43e5b05cbcd9281303d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:11.577905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:11.648303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1446/2891] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:11.653128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.717984 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:11.721759 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a7f41-3fc3-45c1-8a92-97cd59c29faa admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.728953 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.729583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.730070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.730375 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.731535 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.733471 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.762975 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1446/2892] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:11.767316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.779358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb1c43-acce-49cb-934e-58321415f23b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.842673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.843298 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.843580 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.843875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.844548 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.846478 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.870431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1447/2893] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:11 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:11.874646 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.886086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843df84-4bd7-4564-96a2-8034bd6c0c82 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.891239 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.891852 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.892151 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.892422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.893120 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.894720 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.905431 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1447/2894] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:11.909545 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.920458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227c235-f5c1-4d22-b3f3-8bb74c9ca8a9 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:11.925748 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:11.926408 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:11.926709 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:11.926992 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] PATH_INFO: `/v3/projects/6f5ebc4514d943e59f32209c4859b448/users/52ed21b135ce4aafb953a9d1b54c99d2/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:11.944173 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5ebc4514d943e59f32209c4859b448, user_id=52ed21b135ce4aafb953a9d1b54c99d2, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:11.946849 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:11.965240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1448/2895] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:11 2022] PUT /identity/v3/projects/6f5ebc4514d943e59f32209c4859b448/users/52ed21b135ce4aafb953a9d1b54c99d2/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:11.969986 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:11.985704 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69735ad2-7c0f-4395-b21a-556b8072ac80 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.046698 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48c3ce-6a3e-466a-89f8-a235cec634ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.047247 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c3ce-6a3e-466a-89f8-a235cec634ec admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.047533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c3ce-6a3e-466a-89f8-a235cec634ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.047812 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c3ce-6a3e-466a-89f8-a235cec634ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.048537 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48c3ce-6a3e-466a-89f8-a235cec634ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.050146 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48c3ce-6a3e-466a-89f8-a235cec634ec admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.060492 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1448/2896] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:12.065064 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:12.078017 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a52c7-7663-4929-9077-0ea3934d4dc4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.085351 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.086295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.086736 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.087087 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] PATH_INFO: `/v3/projects/6f5ebc4514d943e59f32209c4859b448/users/52ed21b135ce4aafb953a9d1b54c99d2/roles/47aca6aac2934a5cb816872abfc94994` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.089181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5ebc4514d943e59f32209c4859b448, user_id=52ed21b135ce4aafb953a9d1b54c99d2, role_id=47aca6aac2934a5cb816872abfc94994)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.091815 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.102730 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1449/2897] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:12 2022] PUT /identity/v3/projects/6f5ebc4514d943e59f32209c4859b448/users/52ed21b135ce4aafb953a9d1b54c99d2/roles/47aca6aac2934a5cb816872abfc94994 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 05:58:12.113135 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.113446 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.113722 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.156189 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] MFA Rules not processed for user `52ed21b135ce4aafb953a9d1b54c99d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:12.158167 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:12.239703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1449/2898] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:58:12.252984 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:12.254873 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:12.261808 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:12.313928 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd48e8-a2fc-444c-bd8b-b57fd997c95a admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.317821 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.318445 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.318715 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.318990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.328529 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.329969 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.339797 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1450/2899] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 05:58:12.675821 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:12.685043 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b271867-1280-4859-9b40-2820ace31b4e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.690552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ed21b135ce4aafb953a9d1b54c99d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f5ebc4514d943e59f32209c4859b448', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.691149 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.691453 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.691733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.692475 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:12.692780 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.694373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.705713 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1450/2900] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:12.710465 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:12.720330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda411a-cff4-48da-9a8e-024111a606e3 service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.725850 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ed21b135ce4aafb953a9d1b54c99d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f5ebc4514d943e59f32209c4859b448', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.726458 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.726737 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.727008 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.727666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.729602 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.759601 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1451/2901] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:12 2022] POST /identity/v3/projects => generated 353 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:12.765613 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:12.778290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a73d56-4dc3-435e-b764-ccc1f80f520a tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.815635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:12.837355 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ed21b135ce4aafb953a9d1b54c99d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f5ebc4514d943e59f32209c4859b448', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.838003 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.838269 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.838539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.839235 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:12.839539 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.841173 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.853688 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1451/2902] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:12.893104 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:12.904622 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4c64d-31f2-46d4-ba30-4d9795d25c62 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:12.909994 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ed21b135ce4aafb953a9d1b54c99d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f5ebc4514d943e59f32209c4859b448', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:12.910636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:12.910921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:12.911197 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] PATH_INFO: `/v3/projects/5750d10208824e92b8a8d536252eafbc` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:12.912531 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5750d10208824e92b8a8d536252eafbc)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:12.914464 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:12.964794 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5750d10208824e92b8a8d536252eafbc', 'request_id': 'req-c35023d4-ce14-4acb-beee-4f87295b8f71'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:12.970306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5750d10208824e92b8a8d536252eafbc', 'request_id': 'req-c35023d4-ce14-4acb-beee-4f87295b8f71'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:12.977100 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5750d10208824e92b8a8d536252eafbc', 'request_id': 'req-c35023d4-ce14-4acb-beee-4f87295b8f71'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:12.989927 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] The token cache is being invalidate because project 5750d10208824e92b8a8d536252eafbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:12.990212 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:12.990959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-c35023d4-ce14-4acb-beee-4f87295b8f71 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:12.992557 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1452/2903] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:12 2022] DELETE /identity/v3/projects/5750d10208824e92b8a8d536252eafbc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:13.201328 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:13.203179 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:13.271165 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757bc46-883e-4ef4-bc38-721e63e37df7 tempest-VolumeQuotaClassesTest-2059514332 tempest-VolumeQuotaClassesTest-2059514332-project-admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:13.330131 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:13.330784 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:13.331064 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:13.331336 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] PATH_INFO: `/v3/users/2690bdc0630f43e5b05cbcd9281303d5` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:13.332541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2690bdc0630f43e5b05cbcd9281303d5)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:13.334498 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:13.382451 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2690bdc0630f43e5b05cbcd9281303d5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.383196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2690bdc0630f43e5b05cbcd9281303d5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.390715 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2690bdc0630f43e5b05cbcd9281303d5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.391412 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2690bdc0630f43e5b05cbcd9281303d5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.400291 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2690bdc0630f43e5b05cbcd9281303d5'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.413476 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:13.417999 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1452/2904] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:13 2022] DELETE /identity/v3/users/2690bdc0630f43e5b05cbcd9281303d5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:13.422903 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:13.423406 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:13.423496 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:13.442888 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:13.444081 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:13.512534 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1453/2905] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 05:58:13.522445 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:13.524007 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:13.585228 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc285d5d-b812-4649-86b6-cfaf1a6025d2 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:13.590106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:13.590753 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:13.591026 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:13.591301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:13.601062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:13.602553 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:13.612274 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1453/2906] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:13.888733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:13.890561 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:13.903967 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbd817-c105-47fc-a594-04f903b46648 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:13.909542 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:13.911036 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:13.911397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:13.911576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] PATH_INFO: `/v3/projects/463fbe6479e14c799b1a2186bc5e3466` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:13.913086 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=463fbe6479e14c799b1a2186bc5e3466)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:13.915237 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:13.917743 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:13.965742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463fbe6479e14c799b1a2186bc5e3466', 'request_id': 'req-ef81030a-dce2-4469-af0d-b928dd11de62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.971654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463fbe6479e14c799b1a2186bc5e3466', 'request_id': 'req-ef81030a-dce2-4469-af0d-b928dd11de62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.978510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463fbe6479e14c799b1a2186bc5e3466', 'request_id': 'req-ef81030a-dce2-4469-af0d-b928dd11de62'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.990981 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] The token cache is being invalidate because project 463fbe6479e14c799b1a2186bc5e3466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:13.991263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.991913 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:13.996863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1454/2907] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:13 2022] DELETE /identity/v3/projects/463fbe6479e14c799b1a2186bc5e3466 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:13.997989 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.008163 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea0b9d-e942-45d9-a722-dd39e988a8a5 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.065160 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.065792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.066072 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.066348 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] PATH_INFO: `/v3/users/52ed21b135ce4aafb953a9d1b54c99d2` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.067521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ed21b135ce4aafb953a9d1b54c99d2)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.069612 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:14.116568 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ed21b135ce4aafb953a9d1b54c99d2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.117221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ed21b135ce4aafb953a9d1b54c99d2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.124315 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ed21b135ce4aafb953a9d1b54c99d2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.125079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ed21b135ce4aafb953a9d1b54c99d2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.133157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-7f452d53-8526-4cf2-ba2f-7fd1350fabc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ed21b135ce4aafb953a9d1b54c99d2'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.153881 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1454/2908] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:13 2022] DELETE /identity/v3/users/52ed21b135ce4aafb953a9d1b54c99d2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:14.435023 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.436575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:14.450007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81030a-dce2-4469-af0d-b928dd11de62 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.513185 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.514078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.514359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.514636 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] PATH_INFO: `/v3/projects/6f5ebc4514d943e59f32209c4859b448` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.515826 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5ebc4514d943e59f32209c4859b448)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.518039 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:14.577977 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5ebc4514d943e59f32209c4859b448', 'request_id': 'req-b0b64e1f-6aa1-4712-8038-b11f30f730f3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.583427 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5ebc4514d943e59f32209c4859b448', 'request_id': 'req-b0b64e1f-6aa1-4712-8038-b11f30f730f3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.589605 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5ebc4514d943e59f32209c4859b448', 'request_id': 'req-b0b64e1f-6aa1-4712-8038-b11f30f730f3'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.602895 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] The token cache is being invalidate because project 6f5ebc4514d943e59f32209c4859b448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:14.603175 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.603866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-b0b64e1f-6aa1-4712-8038-b11f30f730f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:14.605650 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1455/2909] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:14 2022] DELETE /identity/v3/projects/6f5ebc4514d943e59f32209c4859b448 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:14.610508 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.610993 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.611086 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.631659 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:14.632904 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:14.650067 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:14.694387 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1455/2910] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:14.702423 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.702734 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.703013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.723500 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:14.724669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:14.734215 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1456/2911] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:14.738632 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.748615 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e277d18-0098-4fdd-9e85-dd618691d7e7 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.754275 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.754886 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.755171 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.755448 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.756190 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:14.756499 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.758124 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:14.770861 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1456/2912] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:14.777367 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.791073 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94252fd2-aa14-432b-82c2-c9898a1b04c2 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.796372 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.796973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.797255 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.797553 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.798206 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.800123 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:14.815315 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:14.833696 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1457/2913] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:14 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:14.834813 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.845129 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c58133-8aeb-4ca2-86e0-dc16640a5032 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.906872 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.907488 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.907793 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.908060 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.908723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.910627 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:14.932677 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1457/2914] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:14 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:14.936866 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.947174 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd0cf-0e98-4f69-af02-dfb31419ee05 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.951825 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.952457 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.952738 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.953013 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.953707 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.955316 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:14.967186 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1458/2915] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:14.971278 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:14.982998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e5a8e-ba0d-4371-9cb2-9e985214ce74 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:14.988755 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7cd1c-2747-4642-8d82-a0f06bfa85ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:14.989437 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7cd1c-2747-4642-8d82-a0f06bfa85ac admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:14.989733 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7cd1c-2747-4642-8d82-a0f06bfa85ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:14.990010 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7cd1c-2747-4642-8d82-a0f06bfa85ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:14.990705 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7cd1c-2747-4642-8d82-a0f06bfa85ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:14.992363 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7cd1c-2747-4642-8d82-a0f06bfa85ac admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:15.004285 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1458/2916] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 05:58:15.008665 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:15.019687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1490e64-95a0-44d9-b845-18a6f1863077 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:15.025532 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:15.026202 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:15.026518 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:15.026786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] PATH_INFO: `/v3/projects/be93eefbaac04bafab2a4a7c63436183/users/6eb68ee2ab66481c8e2bfca440ec9121/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:15.042914 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=be93eefbaac04bafab2a4a7c63436183, user_id=6eb68ee2ab66481c8e2bfca440ec9121, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:15.045609 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:15.061345 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1459/2917] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:15 2022] PUT /identity/v3/projects/be93eefbaac04bafab2a4a7c63436183/users/6eb68ee2ab66481c8e2bfca440ec9121/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:15.137595 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:15.137844 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:15.138170 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:15.168029 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] MFA Rules not processed for user `6eb68ee2ab66481c8e2bfca440ec9121`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:15.169260 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:15.242721 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1459/2918] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:15.254647 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:15.256621 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:15.318003 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021fbbc-9f9f-44a7-bf42-ca902ebda1df admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:15.322475 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c764a5-15e7-4afb-a372-c2ccccffd2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:15.323105 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c764a5-15e7-4afb-a372-c2ccccffd2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:15.323389 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c764a5-15e7-4afb-a372-c2ccccffd2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:15.323678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c764a5-15e7-4afb-a372-c2ccccffd2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:15.332694 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c764a5-15e7-4afb-a372-c2ccccffd2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:15.334200 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c764a5-15e7-4afb-a372-c2ccccffd2e0 service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:15.343284 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1460/2919] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:15.364761 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:15.385573 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:15.430916 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564df88a-9c8f-4625-ade3-d7bafdfa859e None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:15.435583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:15.436252 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:15.436533 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:15.436804 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] PATH_INFO: `/v3/users/6eb68ee2ab66481c8e2bfca440ec9121` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:15.437980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb68ee2ab66481c8e2bfca440ec9121)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:15.439940 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:15.494192 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb68ee2ab66481c8e2bfca440ec9121'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:15.495122 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb68ee2ab66481c8e2bfca440ec9121'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:15.504655 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb68ee2ab66481c8e2bfca440ec9121'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:15.505656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb68ee2ab66481c8e2bfca440ec9121'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:15.515253 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb68ee2ab66481c8e2bfca440ec9121'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:15.532306 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1460/2920] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:58:15 2022] DELETE /identity/v3/users/6eb68ee2ab66481c8e2bfca440ec9121 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:15.536996 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:15.537466 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:15.537575 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:15.556804 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:15.558005 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:15.621523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1461/2921] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:15.632063 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:15.633680 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:15.692725 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c35100-94f4-4668-a891-589db1a7e3e3 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:15.696910 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc1b1-4b1c-40d1-912a-c605d0e15630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:15.697522 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc1b1-4b1c-40d1-912a-c605d0e15630 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:15.697801 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc1b1-4b1c-40d1-912a-c605d0e15630 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:15.698081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc1b1-4b1c-40d1-912a-c605d0e15630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:15.707042 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcc1b1-4b1c-40d1-912a-c605d0e15630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:15.708534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcc1b1-4b1c-40d1-912a-c605d0e15630 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:15.718250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1461/2922] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:15.997402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.001671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:16.019324 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415e3c2-fb1b-4239-a8b5-8156ce548a3b None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.024515 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.025412 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.025519 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.025812 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] PATH_INFO: `/v3/projects/be93eefbaac04bafab2a4a7c63436183` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.026989 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be93eefbaac04bafab2a4a7c63436183)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.028953 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.075973 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be93eefbaac04bafab2a4a7c63436183', 'request_id': 'req-250ea2ac-5806-4cfa-b5de-46593c8a78e7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:16.080722 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:16.082891 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be93eefbaac04bafab2a4a7c63436183', 'request_id': 'req-250ea2ac-5806-4cfa-b5de-46593c8a78e7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:16.089631 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be93eefbaac04bafab2a4a7c63436183', 'request_id': 'req-250ea2ac-5806-4cfa-b5de-46593c8a78e7'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:16.102765 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] The token cache is being invalidate because project be93eefbaac04bafab2a4a7c63436183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:58:16.103055 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:16.103727 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-250ea2ac-5806-4cfa-b5de-46593c8a78e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:58:16.105397 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1462/2923] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:58:15 2022] DELETE /identity/v3/projects/be93eefbaac04bafab2a4a7c63436183 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:16.110543 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.110836 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.111114 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.131390 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:16.132641 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:16.169298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:16.196792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1462/2924] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:16.204740 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.205029 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.205303 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.226522 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:16.227863 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:16.236127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1463/2925] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:16.240694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.251200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51f657-569f-4b10-acb5-eb3c75e8b6bd None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.257102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.257723 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.257995 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.258267 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.259036 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80379) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 05:58:16.259346 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.260987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.273134 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1463/2926] 10.222.0.30 () {58 vars in 1153 bytes} [Fri Aug 5 05:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 05:58:16.277595 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.288508 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a255d55-0a15-41c9-8e6b-38cf07186d84 None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.294956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.295581 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.295893 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.296169 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.296843 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.298755 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.328365 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1464/2927] 10.222.0.30 () {60 vars in 1153 bytes} [Fri Aug 5 05:58:16 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 05:58:16.332484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.343694 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396b012-0445-4e70-8083-e40f951e94f5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.402673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.403297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.403574 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.403874 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.404530 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.406444 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.428470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1464/2928] 10.222.0.30 () {60 vars in 1144 bytes} [Fri Aug 5 05:58:16 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 05:58:16.432936 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.443103 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794db9a-f97d-4b5c-a200-d4e94b7e16c4 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.447786 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.448402 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.448678 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.448944 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] PATH_INFO: `/v3/roles` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.449655 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.451268 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.461144 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1465/2929] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:58:16.465004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.477462 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ff460-30f9-4f54-9c8d-e5ce7f085ff5 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.485748 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aae81-a1e3-4f6b-a9a3-f559fcd635f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.486822 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aae81-a1e3-4f6b-a9a3-f559fcd635f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.487128 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aae81-a1e3-4f6b-a9a3-f559fcd635f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.487486 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aae81-a1e3-4f6b-a9a3-f559fcd635f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.488673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aae81-a1e3-4f6b-a9a3-f559fcd635f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.491065 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aae81-a1e3-4f6b-a9a3-f559fcd635f4 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.503255 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1465/2930] 10.222.0.30 () {58 vars in 1122 bytes} [Fri Aug 5 05:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 05:58:16.507455 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.518097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12da1c-b6f9-4f5c-903f-8fe7173ef4de admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.523510 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.524276 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.524563 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.524822 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] PATH_INFO: `/v3/projects/d9c62c503f4844babfd2528fa8c0722f/users/947f7daa83a94adbad8ef6d4a737eccf/roles/3f8debab12324e0da238b80499b5eafe` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.539810 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c62c503f4844babfd2528fa8c0722f, user_id=947f7daa83a94adbad8ef6d4a737eccf, role_id=3f8debab12324e0da238b80499b5eafe)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.542520 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.559184 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1466/2931] 10.222.0.30 () {60 vars in 1483 bytes} [Fri Aug 5 05:58:16 2022] PUT /identity/v3/projects/d9c62c503f4844babfd2528fa8c0722f/users/947f7daa83a94adbad8ef6d4a737eccf/roles/3f8debab12324e0da238b80499b5eafe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:58:16.564440 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d2772-ef3c-4b34-85e7-ce888cb4c3a7 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.564730 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d2772-ef3c-4b34-85e7-ce888cb4c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.564998 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d2772-ef3c-4b34-85e7-ce888cb4c3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.584965 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2b6d2772-ef3c-4b34-85e7-ce888cb4c3a7 None None] MFA Rules not processed for user `c8b08b7805764c45aae169c453594d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:16.586105 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2b6d2772-ef3c-4b34-85e7-ce888cb4c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:16.647470 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1466/2932] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:16.657536 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:16.659384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:16.716385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd53f37-a8c5-4935-a765-495d144b4e06 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:16.720739 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:16.721358 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:16.721666 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:16.721938 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:16.731858 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:16.733306 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:16.742904 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1467/2933] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:17.386516 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036e61-9e6c-4ce5-8508-034057f4397d None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:17.386776 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036e61-9e6c-4ce5-8508-034057f4397d None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:17.387079 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036e61-9e6c-4ce5-8508-034057f4397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:17.414814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-2d036e61-9e6c-4ce5-8508-034057f4397d None None] MFA Rules not processed for user `947f7daa83a94adbad8ef6d4a737eccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:17.416117 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2d036e61-9e6c-4ce5-8508-034057f4397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:17.449166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:17.482360 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1467/2934] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:17.493181 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:17.494599 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:17.551687 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aaba9-a139-4189-b705-2cdd442782b2 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:17.555407 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea0805461cd3492fa5e2ebbea407e351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:17.556128 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:17.556482 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:17.556829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:17.565483 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:17.566933 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:17.575898 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1468/2935] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:18.962482 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:18.963032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:18.963032 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:18.982362 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] MFA Rules not processed for user `947f7daa83a94adbad8ef6d4a737eccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:18.983555 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:18.991210 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1468/2936] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:19.571751 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:19.573226 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:19.608603 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:19.649221 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653e40-5278-41e6-a1b7-39aa2ab4a26b service cinder] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:19.654470 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:19.655092 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:19.655405 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:19.655681 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:19.657498 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:19.668583 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:19.670166 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d2fd7-fa52-47fc-99d4-8540a02f63a9 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:19.680127 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1469/2937] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:25.613289 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:25.614932 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:25.690596 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ffb3-c294-4d84-adca-1573396d83a0 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:25.696016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:25.696636 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:25.696968 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:25.697200 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:25.699250 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:25.710924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:25.712521 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:25.722684 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1469/2938] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:29.676059 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=20 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:32.132263 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d982a-cea2-4a8e-8370-89e92943be45 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:32.132905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d982a-cea2-4a8e-8370-89e92943be45 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:32.132905 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d982a-cea2-4a8e-8370-89e92943be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:32.153908 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-7e6d982a-cea2-4a8e-8370-89e92943be45 None None] MFA Rules not processed for user `947f7daa83a94adbad8ef6d4a737eccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:32.155118 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7e6d982a-cea2-4a8e-8370-89e92943be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:32.163434 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1470/2939] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:58:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:32.405510 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:32.416263 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225c8d-6454-4a2b-a356-000071bee65e service nova] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:32.421249 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:32.421864 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:32.422138 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:32.422406 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:32.424608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:32.434913 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:32.436376 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:32.444463 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1470/2940] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:58:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:58:42.148829 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=3 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:42.440855 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=54 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:58:51.397574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f20e12-bd1a-466c-98d4-a8f379e674c5 None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:51.398057 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f20e12-bd1a-466c-98d4-a8f379e674c5 None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:51.398257 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f20e12-bd1a-466c-98d4-a8f379e674c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:51.461907 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-78f20e12-bd1a-466c-98d4-a8f379e674c5 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:58:51.463136 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-78f20e12-bd1a-466c-98d4-a8f379e674c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:51.530572 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1471/2941] 10.222.0.30 () {58 vars in 1009 bytes} [Fri Aug 5 05:58:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:58:51.580502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:58:51.582098 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:58:51.675400 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee526b-c37c-4043-8152-5d2d2df29228 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:58:51.682188 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e01d7-5522-46ce-89fb-15eda6f04527 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:58:51.682814 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e01d7-5522-46ce-89fb-15eda6f04527 service swift] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:58:51.683092 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e01d7-5522-46ce-89fb-15eda6f04527 service swift] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:58:51.683367 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e01d7-5522-46ce-89fb-15eda6f04527 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:58:51.693428 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e01d7-5522-46ce-89fb-15eda6f04527 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:58:51.695004 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e01d7-5522-46ce-89fb-15eda6f04527 service swift] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:58:51.703211 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1471/2942] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:59:01.528015 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=44 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:59:01.699823 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=41 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:59:03.082192 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] REQUEST_METHOD: `POST` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:03.082756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:03.082756 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:03.104218 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.auth.core [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] MFA Rules not processed for user `4f727e9d632e4592b82ebd1f00cdf8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80378) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:59:03.105617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:59:03.177083 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1472/2943] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 05:59:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:59:06.899583 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:06.900108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:06.900208 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:06.919877 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] MFA Rules not processed for user `8f6cb9dab5f246cabb4d8f1d7f61f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 05:59:06.921084 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:59:06.928422 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1472/2944] 10.222.0.30 () {58 vars in 967 bytes} [Fri Aug 5 05:59:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 05:59:06.941338 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:59:06.953983 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4f623-54e9-4ca7-a3d8-19635bb16dfd None None] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:59:06.959304 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '127c0f72286541159eac49c8f958f816', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:59:06.960025 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:06.960245 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:06.960517 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:06.970286 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:59:06.972024 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:59:06.981291 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1473/2945] 10.222.0.30 () {60 vars in 1384 bytes} [Fri Aug 5 05:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 05:59:08.932529 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:59:08.945534 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9a4f-41fa-4591-97a1-58f003afe278 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:59:08.951423 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:59:08.952078 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:08.952361 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:08.952635 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:08.954506 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:59:08.966502 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:59:08.968132 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:59:08.978257 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1473/2946] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:59:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 05:59:09.027617 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:59:09.061638 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1aa07-3e31-40df-ad0d-33cb0b573393 service swift] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:59:09.067937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:59:09.067937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:09.067937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:09.067937 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] PATH_INFO: `/v3/users/947f7daa83a94adbad8ef6d4a737eccf` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:09.069007 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=947f7daa83a94adbad8ef6d4a737eccf)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:59:09.070990 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:59:09.118330 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947f7daa83a94adbad8ef6d4a737eccf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.118959 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947f7daa83a94adbad8ef6d4a737eccf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.126229 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947f7daa83a94adbad8ef6d4a737eccf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.126842 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947f7daa83a94adbad8ef6d4a737eccf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.135390 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947f7daa83a94adbad8ef6d4a737eccf'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.153632 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1474/2947] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 05:59:09 2022] DELETE /identity/v3/users/947f7daa83a94adbad8ef6d4a737eccf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:59:09.329977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:59:09.342493 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2613-b8d5-4a72-a60b-a7120da92707 service placement] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:59:09.403108 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:59:09.403736 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:09.404056 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:09.404329 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] PATH_INFO: `/v3/projects/d9c62c503f4844babfd2528fa8c0722f` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:09.405469 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c62c503f4844babfd2528fa8c0722f)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:59:09.407381 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:59:09.456325 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c62c503f4844babfd2528fa8c0722f', 'request_id': 'req-c8d506ec-4406-4f32-9922-2357b5e5dcfd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.461924 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c62c503f4844babfd2528fa8c0722f', 'request_id': 'req-c8d506ec-4406-4f32-9922-2357b5e5dcfd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.467943 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c62c503f4844babfd2528fa8c0722f', 'request_id': 'req-c8d506ec-4406-4f32-9922-2357b5e5dcfd'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.494087 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] The token cache is being invalidate because project d9c62c503f4844babfd2528fa8c0722f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80379) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 05:59:09.494397 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.495102 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-c8d506ec-4406-4f32-9922-2357b5e5dcfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 05:59:09.496703 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1474/2948] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 05:59:09 2022] DELETE /identity/v3/projects/d9c62c503f4844babfd2528fa8c0722f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 05:59:19.150058 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=67,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:59:19.491617 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=65,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 05:59:52.562720 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 05:59:52.564355 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:59:52.629080 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eff580-f83a-4292-8f56-2af86aff0d4d admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 05:59:52.693803 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 05:59:52.694441 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 05:59:52.694724 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 05:59:52.694999 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 05:59:52.697211 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 05:59:52.707431 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 05:59:52.708915 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 05:59:52.717576 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1475/2949] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 05:59:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:00:02.713875 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=46 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:00:54.494166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] REQUEST_METHOD: `POST` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:00:54.494758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:00:54.494758 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:00:54.518935 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.auth.core [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] MFA Rules not processed for user `4f727e9d632e4592b82ebd1f00cdf8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80379) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 06:00:54.520201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:00:54.589734 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1475/2950] 10.222.0.30 () {58 vars in 958 bytes} [Fri Aug 5 06:00:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 06:00:54.919385 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:00:54.920997 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:00:54.932170 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fe8f8-d239-4f37-9742-0d5919752104 service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:00:54.937032 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:00:54.937639 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:00:54.937921 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:00:54.938195 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:00:54.940384 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:00:54.950194 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:00:54.951640 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:00:54.959524 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1476/2951] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:00:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:01:04.581221 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=25 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:01:04.956347 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=6 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:02:34.884062 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:34.886579 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:34.902339 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58326297-5a1b-41df-ae50-1ec7bbdce7e3 None None] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:34.906865 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:34.907484 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:34.907796 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:34.908115 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:34.910155 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:34.921297 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:34.922719 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:34.931803 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1476/2952] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:02:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:02:35.782011 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:35.783662 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:35.879336 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef220eb-c079-4bf8-9874-76726a903dce service neutron] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:35.886240 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:35.886872 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:35.887158 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:35.887530 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:35.889352 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:35.907160 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:35.910443 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:35.923851 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1477/2953] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:02:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:02:37.155157 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:37.164648 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dda1f-40b6-4aa2-af70-0609398fd2da service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:37.169316 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:37.170013 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:37.170308 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:37.170608 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:37.172712 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:37.187070 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:37.188545 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:37.200366 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1477/2954] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:02:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:02:45.003178 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:45.005656 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:45.021357 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973cdbb-f0db-4316-8a8d-24a235222343 service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:45.027540 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5134c03bb8854cd4bee58644e6eacc51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:45.028198 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:45.028634 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:45.028948 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:45.031028 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:45.044654 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:45.046290 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:45.057419 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1478/2955] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:02:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:02:47.194907 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=12 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:02:48.169581 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:48.178673 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda2c18-d0b7-4eee-b5e3-d7edebdf4c18 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:48.182373 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfb671f152cd4c40825a81eec15af784', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:48.182923 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:48.183201 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:48.183473 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:48.185212 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:48.256558 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:48.257977 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:48.267507 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1478/2956] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 06:02:51.360725 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:51.365661 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:51.447380 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b6fa6-0859-41fe-8b4d-77fcb40b937f service nova] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:51.452865 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:51.453501 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:51.453781 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:51.454097 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:51.455934 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:51.469476 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:51.471193 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:51.483742 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1479/2957] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:02:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:02:54.656016 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:54.657853 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:54.680290 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c816845-444c-40e8-a7bc-216ec6431229 service neutron] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:54.691196 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:54.691899 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:54.692222 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:54.692528 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] PATH_INFO: `/v3/users/4f727e9d632e4592b82ebd1f00cdf8df` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:54.694081 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f727e9d632e4592b82ebd1f00cdf8df)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:54.696166 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:54.747640 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f727e9d632e4592b82ebd1f00cdf8df'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:54.748319 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f727e9d632e4592b82ebd1f00cdf8df'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:54.755443 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f727e9d632e4592b82ebd1f00cdf8df'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:54.756083 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f727e9d632e4592b82ebd1f00cdf8df'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:54.766441 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f727e9d632e4592b82ebd1f00cdf8df'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:54.787699 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1479/2958] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 06:02:54 2022] DELETE /identity/v3/users/4f727e9d632e4592b82ebd1f00cdf8df => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 06:02:54.964342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:54.984778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49cfb-8503-445b-9f93-546b8e3305e2 service placement] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:55.029577 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=100 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:02:55.053861 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:55.054535 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:55.054795 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:55.055066 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] PATH_INFO: `/v3/projects/d93689b1aaa347368f1f590eb6b494cb` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:55.056410 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93689b1aaa347368f1f590eb6b494cb)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:55.058346 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:55.109227 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93689b1aaa347368f1f590eb6b494cb', 'request_id': 'req-3281671b-f514-4318-9840-f8d38c697cc6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.115261 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93689b1aaa347368f1f590eb6b494cb', 'request_id': 'req-3281671b-f514-4318-9840-f8d38c697cc6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.121732 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93689b1aaa347368f1f590eb6b494cb', 'request_id': 'req-3281671b-f514-4318-9840-f8d38c697cc6'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.136671 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] The token cache is being invalidate because project d93689b1aaa347368f1f590eb6b494cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 06:02:55.136956 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.137562 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.139078 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1480/2959] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 06:02:54 2022] DELETE /identity/v3/projects/d93689b1aaa347368f1f590eb6b494cb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 06:02:55.145121 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:55.146944 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:55.160738 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4cc3-3c43-4dfb-aeab-58ebb1c51978 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:55.223552 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:55.224232 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:55.224541 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:55.224792 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] PATH_INFO: `/v3/users/17c523f9acaa4444b706ef51559031c6` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:55.225990 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c523f9acaa4444b706ef51559031c6)` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:55.227959 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:55.229382 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:02:55.279298 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c523f9acaa4444b706ef51559031c6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.279980 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c523f9acaa4444b706ef51559031c6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.286671 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c523f9acaa4444b706ef51559031c6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.287301 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c523f9acaa4444b706ef51559031c6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.296656 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.notifications [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c523f9acaa4444b706ef51559031c6'} {{(pid=80379) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.315106 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1480/2960] 10.222.0.30 () {58 vars in 1224 bytes} [Fri Aug 5 06:02:55 2022] DELETE /identity/v3/users/17c523f9acaa4444b706ef51559031c6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 06:02:55.511669 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:02:55.515673 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:02:55.533210 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281671b-f514-4318-9840-f8d38c697cc6 admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:02:55.589523 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8b08b7805764c45aae169c453594d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ea26c15401c49d28fd797d4ee85aa6c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:02:55.590179 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:02:55.590459 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:02:55.590729 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] PATH_INFO: `/v3/projects/3b1cb7bfce7c429e907c367ea58ff694` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:02:55.592026 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1cb7bfce7c429e907c367ea58ff694)` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:02:55.593965 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:02:55.642698 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1cb7bfce7c429e907c367ea58ff694', 'request_id': 'req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.648360 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1cb7bfce7c429e907c367ea58ff694', 'request_id': 'req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.654442 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1cb7bfce7c429e907c367ea58ff694', 'request_id': 'req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e'} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.668513 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] The token cache is being invalidate because project 3b1cb7bfce7c429e907c367ea58ff694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80378) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 06:02:55.668733 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.669359 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.notifications [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80378) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 06:02:55.670868 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1481/2961] 10.222.0.30 () {58 vars in 1233 bytes} [Fri Aug 5 06:02:55 2022] DELETE /identity/v3/projects/3b1cb7bfce7c429e907c367ea58ff694 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 06:03:01.822104 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Authenticating user token {{(pid=80379) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:03:01.823785 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:03:01.926639 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b4a31-20ca-4996-b163-7e6fbd17c014 admin admin] Validating token access rules against request {{(pid=80379) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:03:01.976987 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:03:01.992073 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80379) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:03:01.992711 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] REQUEST_METHOD: `GET` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:03:01.992992 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] SCRIPT_NAME: `/identity` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:03:01.993270 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80379) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:03:01.995425 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.fernet_utils [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80379) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:03:02.007299 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:03:02.009103 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4afd55-c9b0-416a-9e81-0ae8b4c8892d service placement] RBAC: Authorization granted {{(pid=80379) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:03:02.019887 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: [pid: 80379|app: 0|req: 1481/2962] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:03:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 06:03:05.665574 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG dbcounter [-] [80378] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80378) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:03:12.015143 dv-cinder-815582-14-iscsi devstack@keystone.service[80379]: DEBUG dbcounter [-] [80379] Writing DB stats keystone:SELECT=8 {{(pid=80379) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 06:08:05.823342 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Authenticating user token {{(pid=80378) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 06:08:05.824935 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.fernet_utils [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80378) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 06:08:05.877051 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdd9e7-ae3c-4d03-a85f-bc128b60b93e admin admin] Validating token access rules against request {{(pid=80378) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 06:08:05.882778 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3758fc-75d7-4189-9277-b63d1dec2410 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b3c6d4d4371427ca4ca74a962a47967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '434c02e53e1341a49390f0aec50efc19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80378) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 06:08:05.883386 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3758fc-75d7-4189-9277-b63d1dec2410 service placement] REQUEST_METHOD: `GET` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 06:08:05.883663 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3758fc-75d7-4189-9277-b63d1dec2410 service placement] SCRIPT_NAME: `/identity` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 06:08:05.883962 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3758fc-75d7-4189-9277-b63d1dec2410 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80378) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 06:08:05.892730 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3758fc-75d7-4189-9277-b63d1dec2410 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 06:08:05.894295 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3758fc-75d7-4189-9277-b63d1dec2410 service placement] RBAC: Authorization granted {{(pid=80378) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 06:08:05.904616 dv-cinder-815582-14-iscsi devstack@keystone.service[80378]: [pid: 80378|app: 0|req: 1482/2963] 10.222.0.30 () {60 vars in 1365 bytes} [Fri Aug 5 06:08:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)